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SECTION  1 
INTRODUCTION 


This  report  documents  the  results  achieved  during  FY  79  on  Project  5220, 
task  522F,  Software  Acquisition  Process  Model.  Since  this  is  the  final  FY  79 
report  on  this  project,  it  includes  previously  reported  findings  to  the  extent 
necessary  to  make  this  report  self-contained. 

The  concept  of  an  ESD  Software  Acquisition  Process  Model  grew  out  of 
earlier  work  in  software  cost  estimation.  Most  of  this  prior  work  was  based 
on  an  algorithmic  or  analytic  approach  by  which  computer  program  attributes 
(such  as  size,  function,  application,  and  complexity)  along  with  developmental 
attributes  (such  as  language,  tools,  methods,  and  experience)  could  be 
converted  into  cost  estimates.  As  reported  in  a  MITRE  review  of  software  cost 
estimation  methods  (CLAPJ76)  the  results  of  using  these  estimation  methods  on 
large  military  systems  have  been  less  than  adequate. 

More  recently,  a  concept  evolved  which  suggested  that  the  software 
development  process  itself  (rather  than  just  overall  attributes  of  the 
software  and  development  methods)  should  be  used  as  a  basis  for  software 
estimating.  While  this  approach  might  add  some  complexity  to  the  estimating 
process,  it  appeared  to  have  compensating  advantages.  As  the  idea  was 
evaluated,  it  became  clear  that  on  military  systems  involving  embedded 
software,  the  development  process  was  too  closely  joined  to  acquisition 
procedures  for  the  two  to  be  separated.  For  this  reason,  we  decided  that  the 
overall  acquisition  process  (including  both  contractor  and  government 
activities)  was  a  better  basis  for  software  estimation  than  development  alone. 
A  succinct  statement  of  this  idea  and  its  rationale  was  provided  in  a  July 
1978  MITRE  letter  to  ESD;  the  relevent  portion  of  this  is  reproduced  in 
Appendix  C,  Rationale  for  Process  Model  Development.  This  approach  was  later 
recommended  within  the  Research  Management  Plan  developed  by  the  Air  Force 
Systems  Command  Software  Cost  Estimating  Working  Group  (SCEWG79).  As  stated 
in  the  letter,  the  MITRE  evaluation 

"strongly  indicates  substantial  potential  benefits  from 
developing  and  applying  in  selected  ESD-managed  acquisition 
programs  a  simulation  model  of  the  software  acquisition  process ■ 

The  type  of  software  acquisition  process  model  considered  would 
represent  explicitly  (e.g.,  in  flowchart  form)  the  different 
Program  Office  (PO)  and  contractor  activities  that  ESD-managed 
software  acquisition  entails,  and  how  these  different  activities 
interact.  It  would  represent  activity  sequences,  repetitions  of 
such  sequences,  alternatives,  concurrency,  and  delays  (e.g.,  due 
to  waits  for  essential  inputs).  In  this  respect  the  model  would 
somewhat  resemble  PERT,  but  without  PERT's  restrictions  on  loop 
representation,  etc." 

The  concept  involved  early  Model  development  and  application  to  ESD- 
managed  Electronic  system  programs,  conducted  per  AFR  800-series  regulations, 
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that  include  acquisition  of  embedded  software.  The  clear  need  for  improved 
ability  to  estimate  better  such  systems'  costs  and  schedules,  and  to  manage 
better  their  development,  guided  the  choice  of  this  general  application  area. 
Later,  the  Model  could  be  revised  or  extended  for  application  to  other  AFSC 
Product  Division  programs,  and  to  other  types  of  systems.  Another  reason  for 
the  early  emphasis  was  availability  at  MITRE  of  persons  with  extensive 
practical  experience  on  such  programs,  able  to  develop  the  Model. 


1.1  POTENTIAL  USES 

A  number  of  advantages  were  seen  for  the  Simulation  Model.  These 
include: 

a.  Improved  accuracy.  This  would  result,  in  part,  from  the  explicit 
contractual  situation  on  which  the  Model  would  base  its  estimates. 

b.  Measures  of  uncertainty,  because  the  Simulation  Model  could  produce  a 
measure  of  estimates'  dispersion,  and  corresponding  estimate  ranges ,  as  well 
as  point  values. 

c.  More  credibility,  because  the  estimates  would  be  based  on  defined 
activities  to  which  an  acquisition  program's  management  could  relate  and  which 
they  could  understand. 

d.  More  flexibility,  because  the  modeled  process  could  include  the 
effects  of  changes  in  development  technique  which  will  occur  as  the  practice 
of  software  engineering  matures. 

e.  More  versatility,  because  the  Simulation  Model  could  support  many 
other  uses  (described  below)  than  the  generation  of  cost  and  time  estimates. 

In  fact,  the  concept  and  simulation  program  (i.e.,  the  Simulator)  could  be 
applied  to  equipment  procurement,  total  system  acquisition,  acquisitions 
conducted  per  different  regulations,  and  many  other  processes. 

While  the  main  driving  force  behind  the  Simulation  Model  is  the  need  for 
better  software-related  cost  and  schedule  estimates,  the  Model  could  be 
effectively  employed  for  other  purposes,  such  as  the  following: 

a.  The  diagrams  of  the  acquisition  process  would  be  useful  for  training 
military  and  support  personnel  for  work  on  software  acquisition.  The 
simulation  program  (i.e.,  the  Simulator)  itself  would  also  help  the  training, 
by  presenting  a  dynamic  picture  which  would  illustrate  the  effects  and 
consequences  of  alternative  actions  and  decisions. 

b.  The  diagrams  would  be  very  helpful  in  project  planning.  They  would 
provide  a  checklist  that  insured  that  important  activities  and  products  were 
not  overlooked,  and  that  contractual  events  and  products  were  scheduled 
realistically;  i.e.,  in  conformance  with  the  organic  needs  of  the  acquisition 
process.  Past  experience  on  many  projects  indicates  that  this  need  has  often 
been  overlooked,  with  dire  consequences.  The  Simulator  would  also  improve  the 
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validity  of  the  system  planning  trade-off  analyses  that  are  performed  to 
establish  the  capability  and  capacity  mix  for  a  particular  procurement. 


c.  The  Simulator  could  be  useful  for  evaluating  contractor  proposals  to 
determine  the  extent  to  which  the  proposed  schedule,  allocated  costs, 
development  plans,  etc.  were  consistent  with  Simulator  forecasts,  and  thus 
with  previous  ESD  experience. 

d.  During  contract  monitoring,  the  Simulator  would  help  evaluate  the 
consequences  of  milestone  slippage,  delays  induced  by  Engineering  Change 
Proposals  (ECPs),  ongoing  cost  reports  vs.  developmental  progress,  etc. 

e.  After  the  Model  was  put  into  routine  use  on  ongoing  contracts,  and  as 
the  data  associated  with  real  contracts  accumulated,  the  Model  accuracy  would 
improve  as  its  processes  and  parameter  values  more  closely  reflected  those 
found  on  real  projects.  As  this  happened  the  Model  would  evolve  in  concert 
with  the  improvements  in  the  software  development  art.  The  resultant 
parameter  changes  would  then  provide  an  objective  measure  of  the  "trend  line" 
and  would  enable  more  accurate  future  forecasts.  At  the  same  time,  the  data 
obtained  on  ongoing  projects  would  enable  the  developmental  performance  on 
different  contracts  to  be  objectively  and  numerically  compared. 

f.  The  graphic  description  of  the  acquisition  process  provided  by  the 
Model  would  present  a  compact  yet  detailed  view  of  how  the  Air  Force  obtains 
embedded  software.  This  view  could  improve  understanding  of  the  process,  and 
thus  help  to  determine  ways  by  which  it  could  be  improved.  The  objectives 
sought,  for  example,  could  be  ways  to  reduce  the  overall  time  or  costs,  to 
obtain  a  more  reliable  product,  or  to  establish  the  cumulative  impact  of  the 
various  system  constituents  (including  operating  functions,  support  functions, 
and  data  items)  for  use  in  tradeoff  studies  that  would  also  consider  each 
constituent's  utility  value.  Use  of  the  Simulator  would  allow  the  dynamics  of 
the  process  to  be  assessed  and  would  make  it  practical  to  obtain  quantitative 
answers  to  complex  questions  for  both  general  acquisition  policy  and  specific 
procurements . 

g.  Finally,  the  Model  could  be  used  as  a  research  tool  for  investigating 
developmental  alternatives  and  managerial  strategies.  It  could  forecast,  for 
example,  the  impact  of  different  manning  assignments,  more  or  fewer 
development  support  facilities,  longer  or  shorter  schedules,  etc. 


1.2  PHASED  DEVELOPMENT  APPROACH 

An  early  objective  of  the  work  on  this  project  was  to  establish  a  balance 
between  the  scope  of  the  work  and  the  resources  that  could  be  made  available 
for  its  accomplishment.  The  potential  width  of  the  Model  was  seen  as  the 
entire  Major  System  acquisition  process  from  the  Conceptual  Phase  through  the 
Deployment  Phase.  The  potential  depth  of  the  work  was  limited  only  by  what 
could  practically  be  described,  estimated,  and  ultimately  simulated. 

The  achievement  of  the  Model's  full  potential  was  taken  as  the  objective 
of  a  multi-year  effort.  Scope  limitations  were  necessary,  therefore,  to  bound 
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the  iuitial  phase  of  the  work.  Some  balance  between  width  and  depth  needed  to 
be  struck. 

While  it  is  a  functional  necessity  for  definition  to  precede 
implementation,  it  is  a  practical  necessity  that  the  consequences  associated 
with  various  implementation  techniques  be  allowed  to  influence  the  style  and 
content  of  the  definition  work.  In  order  to  achieve  adequate  depth,  it  became 
necessary  to  restrict  the  width.  Since  the  width  potentially  includes  all 
five  Acquisition  Life  Cycle  phases,  the  selection  of  at  least  one  of  these 
phases  for  the  initial  work  was  indicated.  After  some  evaluation,  the  Full- 
Scale  Development  (FSD)  Phase  emerged  as  a  logical  choice  for  the  following 
reasons : 

a.  Full-Scale  Development  on  a  system  does  not  ordinarily  begin  until 
requirements  have  been  established  and  an  implementation  concept  has  been 
selected  and  validated.  This  prior  work  provides  much  of  the  basis  for  sizing 
the  software,  and  is  accordingly  an  essential  prerequisite  to  accurate 
costing.  Thus,  more  accurate  estimates  could  be  made  at  the  start  of  the  FSD 
Phase  than  for  earlier  phases  of  the  Acquisition  Life  Cycle. 

b.  The  decision  to  embark  on  Full-Scale  Development  involves  a  high  cost 
commitment  in  which  software  cost  and  timeliness  are  important  considerations. 
Therefore,  since  the  need  for  better  estimates  is  most  acute  at  this  point,  a 
FSD  model  seems  most  likely  to  find  an  early  pilot  application  on  an  ESD 
project. 

c.  Since  FSD  is  the  life  cycle  period  during  which  the  expenditure  rate 
is  highest,  a  FSD  model  should  achieve  the  highest  payoff  to  ESD. 

d.  Finally,  since  the  staff  assigned  to  this  project  were  most 
experienced  with  the  FSD  phase,  they  were  better  able  to  efficiently  model 
that  phase  than  any  other  during  the  critical  Model  formulation  period. 

It  was  also  necessary  to  limit  the  depth  of  the  project  during  FY  79. 

This  was  done  by  establishing  an  evolutionary  development  approach  in  which 
the  first  step  would  be  basic  core  capability.  This  does  not  mean  a 
simplistic  "show  something"  product;  it  means,  instead,  one  that  provides  for 
meaningful  operation  at  a  basic  level,  and  is  designed  to  accommodate  the 
anticipated  future  growth. 

To  guide  this  growth,  several  successive  versions  of  the  Model  were 
defined,  and  a  plan  for  their  phased  development,  pilot  testing,  and 
operational  use  was  outlined.  To  help  implement  this  plan,  we  were  fortunate 
to  devise  a  very  powerful  and  flexible  Simulator  design  concept,  which  will 
assure  a  relatively  small  computer  program,  easy  to  maintain  and  modify. 


1.3  ORGANIZATION 

This  report  has  been  organized  into  a  report  body  that  is  supported  by  a 
number  of  appendices.  The  appendices  generally  serve  to  retain  the 
documentation  developed  during  the  ongoing  definition  and  design  activities. 


They  incorporate  the  main  technical  substance  of  the  work  performed  to  date. 
While  these  appendices  are  too  detailed  for  inclusion  in  the  report  body,  they 
do  provide  important  reference  and  backup  materials  that  have  not  been 
distributed  previously.  For  example,  Appendix  A,  Process  Flow  Diagrams  and 
Amplification  Notes,  especially  Figure  A-2,  Software  Acquisition  Process  Flow 
Diagram  -  LoSim  Level,  depicts  the  entire  FSD  Process,  at  the  level  planned 
for  initial  simulation.  This  figure  may  be  inspected  to  obtain  considerable 
insight  into  this  Acquisition  Life  Cycle  phase.  While  most  readers  are 
probably  familiar  with  (or  have  participated  in)  the  FSD  process,  the  diagram 
can  impart  an  integrated  view  of  the  whole  procedure.  The  overall  complexity 
and  degree  of  interaction  of  the  process  are  not  so  apparent  when  it  is 
experienced  during  the  two  or  three  years  during  which  it  normally  unfolds. 

Similarly,  Appendix  B,  Model  Definition  Data,  contains  estimates  of  the 
manpower  and  elapsed  times  necessary  to  complete  each  of  the  activities 
depicted  in  Figure  A-2,  and  the  probabilities  of  the  decisions  shown  there. 
Appendix  B  may  also  be  of  interest  because  it  represents  the  Figure  A-2  flow 
diagram  network  in  tabular  form  for  use  by  the  Simulator. 

The  appendices  further  show  much  of  the  progress  achieved  during  the  year 
and  the  degree  to  which  the  design  has  matured. 

The  report  body  summarizes  and  ties  together  the  technical  information 
contained  in  the  appendices.  For  those  persons  whose  interest  in  this  work  is 
casual,  a  summary  is  provided  in  Section  2.  Section  3  describes  the  overall 
development  approach.  Sections  4,  5,  and  6  describe,  respectively,  the 
techniques  used  to  define,  quantify,  and  simulate  the  process.  Section  7 
summarizes  the  accomplishments  during  FY  79,  and  the  status  of  the  project  at 
the  end  of  that  fiscal  year.  Section  8  describes  the  areas  of  anticipated 
growth,  and  a  plan  for  its  achievement.  Conclusions  and  recommendations  are 
provided  in  Section  9. 

The  report  contains  numerous  figures  and  tables.  Each  of  these  is 
located  at  the  end  of  the  section  to  which  it  is  most  pertinent.  References 
to  figures  and  tables  in  the  body  of  the  report  which  are  not  part  of  a 
section  include  page  numbers  to  facilitate  access.  The  List  of  Illustrations 
can  be  used  to  locate  all  figures  and  the  List  of  Tables  all  tables. 


SECTION  2 


SUMMARY 


Virtually  all  major  military  systems  now  include  computers  and  associated 
software  as  essential  elements  for  providing  system  functions.  As  these 
systems  have  evolved,  the  computer  programs  (termed  embedded  software)  have 
grown  in  capability  and  size.  They  have  also  contributed  greatly  to  system 
development  costs  and,  all  too  often,  to  cost  overruns  and  slipped  schedules. 

For  these  reasons,  project  planners  need  a  method  of  obtaining  reliable 
estimates  for  the  cost  and  time  of  acquiring  embedded  software.  While  many 
methods  are  in  current  use,  none  have  produced  results  with  sufficient 
accuracy  and  reliability  to  meet  needs. 

Recently,  a  concept  evolved  which  suggested  that  more  accurate  estimates 
could  be  obtained  by  using  a  simulator  to  model  the  process  by  which  software 
is  developed  and  acquired.  By  decomposing  the  overall  process  into  unit 
functions  which  could  be  grasped  and  evaluated,  and  by  allowing  development 
decisions  (and  resulting  rework)  to  be  explicitly  included,  more  accurate 
results,  plus  expected  variations,  would  be  obtained. 

A  project  based  on  this  concept  was  funded.  The  results  of  the  FY  79 
work  are  described  in  this  repo'rt.  The  software-related  work  in  the  Full- 
Scale  Development  Phase  of  the  Major  System  acquisition  process  has  been 
carefully  defined  by  means  of  several  levels  of  process  flow  diagrams  plus 
supplemental  information.  The  process  has  been  quantified  to  express  the  cost 
and  time  contributions  of  each  unit  process  as  well  as  the  probability  value 
associated  with  each  decision  element.  A  simulation  concept  was  developed  by 
which  the  whole  acquisition  process  could  be  converted  into  tabular  form  and 
then  used  to  drive  a  rather  small  computer  program  that  conducts  the 
simulation  and  collects  the  results.  The  actual  Simulator  design  is  well 
along,  and  some  of  it  has  been  coded  and  compiled. 

Considerable  progress  was  made  on  a  first  model  of  the  Simulator 
(Model  0).  Future  plans  call  for  a  series  of  successive  models,  each  of  which 
captures  more  of  the  inherent  subtlety  of  the  acquisition  process.  Also  the 
later  models  will  become  more  generic  and  therefore  more  readily  adapted  to  a 
wide  range  of  projects.  Plans  also  include  an  Increase  in  breadth  of  the 
Model  so  that  all  phases  of  the  Major  System  acquisition  process,  from  the 
Conceptual  to  the  Deployment  Phases,  are  included. 

During  FY  79,  much  has  been  accomplished  and  much  learned.  Based  on  that 
experience  the  Process  Model  looks  entirely  feasible  and  the  original  promise 
remains  bright.  Moreover,  a  number  of  other  likely  uses  for  the  Simulator 
have  been  noted;  its  growth  into  a  general  management  tool  for  wide  use  in 
acquiring  embedded  software  appears  promising. 

Due  to  the  results  accomplished,  and  the  potential  value  of  a  Software 
Acquisition  Process  Model  Simulator,  MITRE  recommends  continuing  the  project. 
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SECTION  3 


APPROACH 


The  overall  technical  effort  on  the  project  is  being  channeled  into  three 
principal  areas.  These  work  areas,  which  are  briefly  introduced  below,  are 
detailed  in  Sections  4  through  6,  respectively. 

a.  Process  Definition.  This  work  involves  the  creation  of  Process  Flow 
Diagrams  and  Explanatory  Notes  which  represent  the  process  whereby  computer 
software  is  acquired  by  ESD  under  the  AFR  800-series  regulations.  In 
particular,  the  project  is  focused  on  software  which  is  embedded  within  a 
large  command,  control,  and  communications  system  (see  DoDD  5000.29).  For 
maximum  realism  these  Process  Flow  Diagrams  represent  both  sequential  and 
concurrent  activities.  Otherwise,  to  facilitate  communication,  they  are 
conventional  Von  Neumann  flowcharts. 

b.  Process  Quantification.  This  work  involves  establishing  parameters 
which  describe  each  element  within  the  Model,  and  obtaining  appropriate 
initial  values  for  these  parameters. 

c.  Process  Simulation.  This  task  requires  that  the  Model  be  mechanized 
so  that  it  can  be  used  to  carry  on  synthetically  the  processes  defined,  using 
the  assigned  parameters.  It  can  thereby  forecast  and  report  the  statistical 
consequences  in  terms  of  probable  schedule  and  cost  distributions.  A  discrete 
event  simulation  program  (i.e.,  the  Simulator)  is  the  mechanism  of  choice. 

The  Process  Definition  and  Process  Quantification  work  is  based 
principally  on  the  authors'  personal  acquisition  program  experience.  This  has 
been  supplemented  by  review  of  pertinent  regulations,  specifications, 
standards,  and  data  item  descriptions  (DIDs)  performed  to  prepare  a  series  of 
software  acquisition  management  guidebooks. 

The  three  work  areas  needed  to  be  started  in  the  order  presented. 

However,  because  of  the  interdependence  among  them,  the  work  was  programmed  to 
allow  for  the  changes  induced  in  each  task  by  the  "ripples"  created  by  the 
others.  For  this  reason,  Process  Definition  was  stressed  at  first  until  there 
was  a  basis  for  beginning  the  work  in  the  other  areas.  Later,  work  proceeded 
simultaneously  and  interactively  in  all  three  areas,  but  with  emphasis 
gradually  shifting  to  simulation  program  design. 

Another  consideration  in  this  conceptual  development  concerned  the  level 
of  detail  to  be  included  in  the  Model  and  the  extent  to  which  the  various 
Model  processes  would  interact.  After  some  exploratory  investigation,  an 
evolutionary  approach  was  selected  in  which  a  relatively  simple  version  of  the 
Model  would  be  defined  and  implemented  first;  this  would  be  followed  in  turn 
by  a  sequence  of  increasingly  realistic  versions.  This  evolutionary  approach 
led  to  a  consideration  of  Model  attributes,  their  utility,  and  the  amount  of 
effort  needed  for  their  definition  and  simulation.  There  was  early 
recognition  that  an  overall  lifelike  simulation  of  the  process  would  be 


complex  and  would  need  to  include  many  subtle  interactions.  At  the  same  time, 
it  appeared  that  simpler  versions  could  be  useful,  if  built  for  limited 
applications.  As  a  result,  an  initial  version  of  the  Simulator  (i.e., 

Model  0)  was  defined  and  capabilities  were  established  for  several  successive 
versions.  In  addition,  other  Model  attributes  were  identified  which  could  be 
added  later,  in  response  to  the  needs  and  priorities  of  the  Model  users. 

Despite  the  desire  for  simplicity,  the  initial  Model  includes  capability 
for  dealing  with  basic  design  and  acquisition  concepts  which  are  only 
indirectly  treated  (if  at  all)  by  other  tools.  One  of  these  concepts  is 
phased  implementation  (see  Section  4.1.5).  By  using  this  concept  the  system 
software  developer  does  not  need  to  confront,  design,  and  implement  the 
required  capability  in  its  defined  totality.  Instead,  he  establishes  a  set  of 
versions  which  are  to  be  developed  sequentially  until  the  first  deliverable 
version  is  completed.  This  concept  was  used  with  considerable  success  on  a 
recent  ESD  project  called  SALTY  NET,  and  is  expected  to  receive  widespread 
application  in  the  future,  particularly  on  major  weapon  system  acquisition 
projects.  This  concept  is  also  being  used  informally  on  this  small-scale 
project. 

Interaction  and  iteration  are  also  explicitly  represented  in  this  Model. 
While  these  have  always  plagued  the  ongoing  development  process,  other 
planning  and  estimation  methods  generally  treat  them  indirectly  (e.g.,  via 
loading  factors)  or  ignore  them.  Since  we  believe  that  these  factors  are 
often  largely  responsible  for  the  wide  disparity  between  the  project  plan  and 
its  realization,  the  Model  includes  paths  for  both  forward  and  iterative 
progression. 

In  addition,  the  Model  includes  decision  elements  which  select  among 
exclusive  alternative  paths.  The  Model  also  includes  a  mechanism  for  allowing 
the  results  of  earlier  processes  to  influence  the  consequent  results  of  later 
ones.  While  this  mechanism  (i.e.,  the  Special  Event)  will  receive  limited  use 
in  the  earlier  Simulator  versions,  it  provides  a  basis  for  emulating  much  of 
the  subtlety  of  the  acquisition  process.  Later  versions  are  expected  to  make 
extensive  use  of  this  capability. 

Note  that  the  provisions  for  phased  implementation,  interaction,  decision 
elements,  and  iteration  have  been  costly  in  terms  of  definition  and  design 
effort.  They  have  been  included,  nevertheless,  to  avoid  a  design  which  could 
not  accommodate  the  maturing  needs  of  the  Model  users. 

In  order  for  a  simulator  to  work,  it  is  necessary  that  quantifying 
parameters  be  selected  and  specific  parameter  values  be  applied  to  the 
individual  function  boxes  which  populate  the  Model.  Eventually,  this  will 
need  to  be  accomplished  by  the  development  of  valid  relationships  between  the 
time  expended  in  each  function  box  and  such  factors  as: 

a.  the  quantity  and  quality  of  manpower  and  other  resources  available; 

b.  the  extent  and  difficulty  of  the  task  represented  by  the  box;  and 
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c.  the  size  and  complexity  of  the  total  project  and  of  the  computer 
program  component  or  module  being  addressed  by  the  box. 

The  development  of  these  time  and  cost  relationships  and  their  validation 
require  the  expenditure  of  much  effort  and  time  -  much  more  than  have  been 
available  on  this  project.  Since  little  prior  work  has  been  published  (other 
than  gross  estimates  such  as  those  which  ascribe  cost  distribution,  like  40% 
design,  20%  coding,  and  40%  test),  this  type  of  work  would  need  to  start  from 
basics . 

For  these  reasons,  a  more  limited  approach  was  taken  for  obtaining 
parameter  values.  In  this  approach,  a  set  of  parameter  values  (per  function 
box)  were  developed  for  a  typical  project.  These  values  were  originally 
obtained  from  estimates  made  by  persons  familiar  with  the  individual  processes 
being  modeled.  The  estimates  were  then  calibrated  by  comparing  the  project 
schedule  derived  from  these  estimates  with  a  typical  schedule  for  such  a 
project,  as  obtained  from  military  procurement  experience.  The  original 
parameter  estimates  for  each  segment  of  the  schedule  were  then  scaled  to  make 
the  derived  schedule  coincide  with  the  one  estimated. 


SECTION  4 


PROCESS  MODEL  DEFINITION 


In  this  section  information  is  presented  which  describes  our  modeling  of 
the  software-related  aspects  of  the  Full-Scale  Development  Phase  of  the  Major 
System  acquisition  process.  Paragraph  4.1  describes  the  basic  premises 
followed  as  the  Process  Model  was  being  defined;  Paragraph  4.2  introduces  the 
set  of  diagrams  by  which  the  Model  of  the  process  was  described.  Paragraph 
4.3  describes  some  prior  projects  wherein  the  authors  gained  relevant 
experience. 


4.1  BASIC  PREMISES 

During  preparation  of  the  Software  Acquisition  Process  Model,  it  was 
found  necessary  to  delineate  the  Model  and  to  limit  the  scope  of  the  effort  to 
fit  within  a  limited  budget  and  schedule.  The  set  of  basic  premises  discussed 
below  was  established,  therefore,  as  guidance  for  the  initial  phases  of  this 
work.  Some  of  these  apply  to  the  acquisition  process  itself,  others  to 
simplifications  introduced  for  application  to  early  versions  of  the  Simulator. 
These  premises,  whenever  applicable,  are  referenced  by  Table  A-2,  Process  Flow 
Diagram  Amplification  Notes,  which  supports  the  Process  Flow  Diagrams. 

4.1.1  Conformance  to  Military  Standards 

The  acquisition  process  modeled  will  conform  to  all  military  standards 
and  regulations  that  are  normally  applied  to  software  acquired  during 
Electronic  System  procurements.  These  include  MIL-STD-483,  Configuration 
Management  Practices  for  Equipment,  Munitions,  and  Computer  Programs; 
MIL-STD-1521A,  Technical  Reviews  and  Audits  for  Systems,  Equipment,  and 
Computer  Programs;  AFR  800-2,  Acquisition  Program  Management;  and  AFR  800-14, 
Vol.  II,  Acquisition  and  Support  Procedures  for  Computer  Resources  in  Systems. 
If  deviation  from  these  practices  is  found  to  be  necessary,  it  will  be 
explicitly  described  (and  explained)  at  each  point  in  the  process  where  it 
occurs;  a  summary  list  of  all  such  deviations,  if  any,  will  be  provided. 

4.1.2  System,  Segment,  and  CPCI  Relationships 

The  relationships  among  activities  associated  principally  with  a  system, 
its  segments,  and  its  Computer  Program  Configuration  Items  (CPCIs)  will  be 
considerably  simplified  in  the  early  implementations.  In  particular,  system 
segments  can  be  used  in  different  ways  on  different  contracts  and  are 
therefore  not  fully  amenable  to  generic  implementation.  For  this  reason,  the 
Model  addresses  the  CPCI  (level  3)  and  one  level  higher.  While  this  higher 
level  is  referred  to  as  "system"  (level  1)  it  could  as  readily  represent 
"system  segment"  (level  2).  The  choice  is  dependent  on  the  nature  of  the 
system  and  the  specific  contract(s)  being  simulated. 


In  addition,  while  the  Model  is  designed  to  accommodate  a  number  of 
CPCIs,  it  will  treat  these  initially  in  a  somewhat  simplified  manner.  As  thus 
modeled,  all  CPCIs  will  initiate  and  terminate  together  (e.g.,  in  the  System 
Test),  and  proceed  independently  in  between.  In  actual  practice,  the  various 
CPCIs  often  have  dependency  relationships  which  can  be  of  critical  importance 
to  the  success  of  a  project  (see  Figure  6,  notes  B-D,  page  39).  Later 
versions  of  the  Model  will  be  designed  to  accommodate  these  relationships. 

4.1.3  Validation  Phase  Activities 

The  Process  Model  of  the  Full-Scale  Development  Phase  presumes  that  a 
full  Validation  Phase  has  already  been  completed.  However,  since  many 
projects  omit  this  phase  but  incorporate  some  of  its  activities  in  the  Full- 
Scale  Development  Phase,  provision  should  be  made  for  such  activities' 
incorporation  (e.g.,  the  preparation  of  development  specifications)  in  the  FSD 
Phase  Model.  Extension  of  the  Model  to  the  Validation  Phase  is  planned  for 
later  implementation.  The  process  flow  developed  for  that  phase  will  be 
designed  so  that  selected  activities  can  be  readily  moved  into  the  Full-Scale 
Development  Phase. 

4.1.4  Support  Facilities 

The  Model  will  presume  that  the  Test  and  Programming  Support  functions 
will  each  be  provided  by  separate  facilities.  On  some  projects,  such 
facilities  may  be  shared  (in  whole  or  in  part)  to  support  both  functions.  The 
Model  will  be  designed  to  reflect  any  combined  use  of  these  facilities. 

While  the  current  Model  provides  for  accumulating  the  costs  of  operating 
and  maintaining  support  facilities  and  for  the  impact  resulting  from  their 
late  availability,  it  does  not  include  the  effect  of  contention  between 
facility  users  or  the  results  of  unscheduled  down  time.  These  latter 
capabilities  will  be  added  in  later  versions. 

4.1.5  Phased  Implementation  Provisions 

Procurement  regulations  allow  design  reviews  to  be  conducted  on  a  single 
or  on  an  incremental  basis.  The  Model  is  being  designed  to  represent  the 
incremental  approach.  While  this  decision  adds  to  the  complexity  of  the 
Model,  it  was  taken  because  the  single  design  review  approach  would  not 
support  the  trend  toward  phased  development,  particularly  for  larger  systems. 
The  Model  will  also  accommodate  the  single  design  review  approach,  simply  by 
setting  the  number  of  design  increments  to  one. 

The  initial  Model  is  being  designed  to  accommodate  the  following 
incremental  approach: 

a.  Each  CPCI  is  defined  by  a  specification  which  states  the  functional 
requirements  to  be  met  at  the  completion  of  the  current  procurement  contract. 
While  certain  follow-on  requirements  may  also  be  explicitly  or  implicitly 
defined,  these  are  treated  as  beyond  the  scope  of  that  contract. 
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b.  The  contractor  would  divide  the  total  contractual  requirements  into 
several  increments  (hereafter  called  Developmental  Integration  Groups  (DIGs)). 
This  division  would  be  defined  in  a  phased  implementation  plan  that  is 
included  within  the  Computer  Program  Development  Plan  (CPDP). 

c.  As  shown  in  Figure  5,  Phased  Group  Development  Example,  the 
contractor  would  then  proceed  with  the  design  of  the  first  DIG  (DIG-I).  The 
work  on  this  DIG  would  then  pass  successively  through  the  various  stages  of 
the  design  process  (including  Preliminary  Design  Review  (PDR)  and  Critical 
Design  Review  (CDR)),  and  through  coding,  debugging,  integration,  and 
contractor  internal  testing.  The  work  might  also  be  subject  to  Preliminary 
Qualification  Testing  (PQT) ,  but  not  to  Formal  Qualification  Testing  (FQT) . 

d.  The  design  and  implementation  of  the  other  DIGs  would  proceed  in 
order  behind  DIG-I.  Work  on  the  second  DIG  (DIG-II)  would  begin  after  PDR  on 
DIG-I;  DIG-III  would  start  after  PDR  on  DIG-II,  etc.  Similarly,  the  CDRs  and 
other  development  activities  would  proceed  in  the  same  order. 

e.  During  each  stage  of  development,  each  successive  DIG  would  add  to 
and  build  onto  the  aggregated  preceding  DIGs.  In  other  words,  a  single  CPCI 
would  be  built  in  successive  stages;  it  would  not  be  built  as  separate  DIGs  to 
be  joined  together  at  the  end. 

f.  When  the  last  DIG  passed  through  each  development  stage,  the  total 
implementation  to  that  stage  would  be  complete.  Therefore,  each  last  DIG 
design  review  would  be  extended  to  survey  the  totality  of  the  design,  in 
addition  to  that  of  the  last  functional  increment. 

g.  The  Model  documentation  includes  notation  to  accommodate  the 
incremental  development  concept.  The  notation  will  indicate  (with  a  "D";  see 
Figure  A-l)  those  processes  which  are  presumed  developed  in  this  phased 
manner.  In  addition,  when  a  development  phase  is  complete  for  one  DIG,  the 
process  must  return  to  that  phase  to  begin  work  on  the  next  DIG.  This  type  of 
return  is  shown  as  type  "D"  on  the  Process  Flow  Diagram  (Figure  A-2)  and  in 
the  Network  Definition  Table  (Table  B-l)  (in  its  General  Data  Grp  column). 

h.  The  formal  test  activities  may  also  be  conducted  on  a  similarly 
phased  basis.  The  Model  will  support  this  approach  by  allowing  Test 
Integration  Groups  (TIGs)  to  be  sequentially  processed  in  a  manner  analogous 
to  the  handling  of  DIGs.  Note  that  the  TIG  division  involves  the  test  related 
activities  and  applies  to  a  totally  implemented  CPCI;  therefore,  TIGs  are  not 
related  to  DIGs  in  terms  of  usage  or  quantities. 

4.1.6  Incidental  Activities 


While  the  Model  is  planned  to  include  all  significant  mainstream 
acquisition  activities,  it  will  not  include  a  number  of  incidental  tasks  that 
are  essential  to  a  project  but  that  would  add  needless  complexity  to  the 
Model.  Instead,  the  cost  and  loading  impact  of  such  activities  will  be 
included  as  general  overhead  factors.  Similarly,  certain  events  and 
activities  judged  too  infrequent  or  too  inconsequential  to  the  Model  (though 
not  to  the  acquisition  process)  will  not  be  included.  Should  experience  or 
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collected  data  indicate  that  some  of  these  incidental  activities  be  added  to 
the  Model,  this  can  be  done  in  a  later  version. 


4.1.7  Resource  Utilization 


Each  process  activity  consumes  project  resources  such  as: 

a.  contractor  manpower  in  various  job  categories; 

b.  government  manpower  in  various  job  categories; 

c.  development  support  facilities; 

d.  test  support  facilities; 

e.  miscellaneous  other  resources. 

In  the  early  implementations,  only  manpower  resources  are  being  assigned 
to  specific  process  activities.  There  are  two  reasons  for  this.  First, 
manpower  is  by  far  the  most  important  resource  in  software  acquisition. 

Second,  because  of  this,  we  deemed  it  more  important  to  develop  reasonably 
sound  initial  manpower  estimates  than  to  divert  effort  toward  estimating  other 
resource  requirements.  The  manpower  categories  listed  below  were  selected  for 
implementation  based  on  our  acquisition  program  experience.  In  addition,  the 
manpower  accounting  techniques  and  the  effects  of  resource  limitation  are 
described  below. 

a.  Contractor  personnel.  Five  job  categories  were  selected  for 
individual  assignment  to  each  activity: 

(1)  systems  engineer  or  analyst; 

(2)  designer; 

(3)  programmer; 

(4)  test  engineer; 

(5)  support  (e.g.,  equipment  operator,  librarian,  documenter) . 

During  our  initial  Process  Model  work,  Management  was  included  as  an 
additional  category.  This  separate  Management  category  was  abandoned  when  the 
need  to  subdivide  a  manager's  time  among  many  ongoing  tasks  made  its 
estimation  impracticable.  Aside  from  the  difficulty  in  estimation,  results 
would  be  inaccurate  because  management  styles  differ  widely  and  would 
generally  be  unknown.  For  this  reason,  we  decided  to  represent  management  as 
a  continuous  activity  with  an  overall  manpower  profile  that  conforms  to  the 
estimated  (or  given)  needs  for  the  project  being  modeled. 

b.  Government  personnel.  Three  job  classifications  were  selected  for 
personnel  assignment  to  specific  activities;  these  reflect  the  three  principal 
commands  involved  in  system  acquisition:  The  Developing  Command  (e.g., 
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Electronic  Systems  Division  (ESD)),  the  Using  Command  (e.g.,  Tactical  Air 
Command  (TAC)),  and  the  Supporting  Command  (e.g.,  Air  Force  Logistics  Command 
(AFLC)).  Consideration  was  given  to  further  specifying  the  assignments  (e.g., 
to  Engineering,  Test,  Configuration  Management,  etc.)  but  it  was  decided  that 
this  would  be  more  appropriate  in  later  versions. 

c.  Initial  implementation  technique.  Recognizing  that  personnel 
categories  are  likely  to  differ  for  different  contractors  and  projects,  a 
generic  assignment  technique  was  dictated.  The  method  selected  can  be  used 
for  any  number  of  categories.  While  a  one-dimensional  list  of  manpower 
categories  is  being  implemented,  it  can  be  expanded  to  two  (or  more) 
dimensions  if  needed.  This  would  allow,  for  example,  the  group  of  ESD 
personnel  assigned  to  an  acquisition  program  to  be  further  divided  into 
functional  groups,  such  as  Engineering  or  Test.  It  would  also  allow 
contractor  designers,  for  example,  to  be  further  distinguished  as  senior  or 
junior,  etc.  Working  versions  of  the  Simulator  will  eventually  use  job 
categories  that  are  compatible  with  those  developed  for  the  planned  ESD 
Software  Acquisition  Resource  Expenditure  (SARE)  reporting  system. 

d.  Resource  limitations.  The  rate  of  progress  on  any  project  can  be 
strongly  influenced  by  the  quantity  and  quality  of  the  available  resources. 

The  Simulator,  in  Model  1  and  later,  will  allow  the  amount  of  resources  to  be 
defined  such  that  each  activity  can  draw  from  a  resource  bank  when  it  is  ready 
to  begin.  When  the  demand  for  a  resource  exceeds  its  supply,  the  process  will 
slow  accordingly.  This  process  behavior,  while  inherently  simple,  may  require 
that  different  and  perhaps  complex  management  strategies  be  devised  to  resolve 
automatically  the  problem  of  allocating  scarce  resources  among  competing 
activities.  For  this  reason,  the  initial  Simulator  version  will  not  reflect 
the  effects  of  resource  limitation.  Instead,  it  will  allocate  manpower  only 
on  the  basis  of  the  needs  of  each  activity,  and  keep  track  of  the  amount  used. 
The  end  result  of  a  simulation  run  will  include  a  statistical  profile  of  the 
total  amount  of  manpower  used  in  each  category  versus  time. 


4.2  PROCESS  FLOW  DIAGRAMS 

Process  Flow  Diagrams  have  been  used  as  the  principal  means  for 
describing  the  process  of  acquiring  embedded  software.  They  were  developed  at 
several  levels  of  detail,  as  follows: 

a.  a  global  view  of  the  whole  process; 

b.  a  high  simulation  level  (HiSim); 

c.  a  low  simulation  level  (LoSim);  and 

d.  expanded  views  of  LoSim  boxes  to  show  more  elemental  relationships  as 
needed . 

The  conventions  followed  by  these  Process  Flow  Diagrams  are  described  in 
Appendix  A,  Figure  A-l,  Flow  Diagram  Notation.  Briefly,  they  define  three 
types  of  basic  element:  (1)  function  boxes;  (2)  auxiliary  elements  (e.g., 
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connectors);  and  (3)  lines  of  flow.  These  conventions  should  be  understood 
before  the  Process  Flow  Diagrams  are  reviewed. 


The  LoSim  and  Process  Flow  Expansion  Diagrams  (Figures  A-2  and  A-3, 
respectively)  distinguish  support  activities  from  mainstream  activities  by 
representing  the  former  in  trapezoidal  boxes  while  limiting  the  use  of 
rectangular  boxes  to  the  latter.  This  distinction  is  not  made  in  the  HiSim 
and  higher-level  Process  Flow  Diagrams  (Figures  1-3).  There,  rectangular 
boxes  represent  all  activities. 

4.2.1  Global  View 


The  sequential  relationship  among  the  phases  that  constitute  the  standard 
Major  System  acquisition  process  (defined  in  AFR  800-2,  Acquisition  Program 
Management)  is  shown  in  Figure  1,  Major  System  Acquisition  Process  Normal 
Phases  of  Progression.  The  Full-Scale  Development  Phase  is  emphasized  in  this 
figure  to  indicate  that  it  is  the  focus  of  the  initial  work  being  accomplished 
on  this  project.  The  principal  groups  of  Full-Scale  Development  Phase 
activities  and  the  most  important  lines  of  flow  among  them  are  shown  in 
Figure  2,  Software  Acquisition  Process  Model  Activity  Flow  Overview  -  Full- 
Scale  Development  Phase.  Besides  providing  a  global  view,  this  figure  aids 
access  to  two  more  detailed  diagrams,  termed  simulation  level  diagrams,  that 
the  Simulator  can  mechanize: 

a.  The  box  numbers  in  Figure  2  refer  to  those  used  in  Figure  3,  Software 
Acquisition  Process  Flow  Diagram  -  HiSim  Level.  Two  of  the  Figure  2  box 
numbers  (i.e.,  66  and  96),  which  pertain  to  the  Program  Management  Review 
(PMR)  and  ECP  processes,  respectively,  are  not  shown  in  Figure  3.  Inclusion 
of  these  processes  was  deemed  inappropriate  because  both  can  interact  with 
most  of  the  other  ongoing  activities  in  a  way  which  cannot  be  properly 
represented  at  the  HiSim  level. 

b.  The  sheet  number  references  given  in  each  Figure  2  process  flow 
activity  group  box  refer  to  Figure  A-2,  Software  Acquisition  Process  Flow 
Diagram  -  LoSim  Level.  The  box  numbers  in  Figure  2  also  refer  to  the  digit 
portion  of  the  alphanumeric  box  identifiers  shown  in  Figure  A-2. 

4.2.2  Simulation  Level  Diagrams 

Two  simulation  levels  are  planned  to  support  different  purposes  to  which 
the  Simulator  may  be  put.  The  two  levels  may  be  either  used  separately  or 
intermixed.  This  will  let  detailed  simulation  results  be  obtained  in  selected 
areas  while  the  remaining  portions  of  the  process  are  treated  more  generally. 
Also,  the  whole  process  or  just  a  portion  of  it  may  be  simulated. 

Note  that  transition  from  low  to  high  level  modeling  involves  both 
coalescence  of  many  boxes  into  a  few  and  abridgement  of  the  lines  of  flow 
(i.e.,  network  linkage).  While  box  coalescence  is  readily  accomplished,  the 
need  to  retain  network  continuity  will  require  that  mixed-mode  simulation 
level  transitions  take  place  only  at  points  where  the  lines  of  flow  are 
compatible. 


23 


4.2.2. 1  The  High-level  Simulation  (HiSim)  Diagram 


The  HiSim  diagram  (Figure  3)  views  the_ Full-Scale  Development  Phase  as  a 
serial  and  parallel  sequence  of  32  composite  activities.  This  diagram  also 
shows  the  main  lines  of  process  flow  that  connect  these  activities.  A 
simplified  notation  has  been  used  to  label  those  connectors  that  reflect 
dependencies  between  activities  that  fall  in  different  main  lines  of  flow. 
Thus,  all  connectors  that  lead  to  documentation  activities  have  labels  that 
begin  with  a  "D".  Similarly,  test-related  connector  labels  begin  with  "T". 

The  beginning  and  ending  Terminals  are  labeled  "B"  any  "Y",  respectively,  in 
agreement  with  the  notation  used  on  all  the  other  process  flow  diagrams. 

While  Figure  3  provides  a  modestly  detailed  overview  of  the  FSD  process, 
it  is  important  to  recognize  that  the  following  are  not  represented: 

a.  decisions  (i.e.,  exclusive  alternatives); 

b.  task  iteration; 

c.  Integration  Group  (i.e.,  DIG  and  TIG)  progression; 

d.  most  distinctions  between  contractor  and  government 
activities . 

Because  of  these  omissions,  pure  HiSim  simulation  appears  to  have 
marginal  value  except  for  general  project  planning  and  estimation  before 
further  detail  becomes  available.  If  thus  used,  the  HiSim  parameter  values 
should,  to  be  realistic,  represent  the  results  of  simulation  at  a  lower  level. 

The  HiSim  Process  Flow  Diagram  looks  more  promising  as  the  initial  basis 
for  a  mixed-mode  simulation;  (i.e.,  one  that  includes  more  detail 
selectively).  This  detail  would  come  from  the  LoSim  Process  Flow  Diagram 
(discussed  next)  or  modifications  thereof. 

4. 2. 2. 2  The  Low-level  Simulation  (LoSim)  Diagram 

The  LoSim  Process  Flow  Diagram  (Figure  A-2)  uses  approximately  two 
hundred  boxes  on  twelve  pages  to  represent  the  overall  process.  The  function 
of  each  box  is  described  in  abbreviated  English,  but  box  size  limitations  make 
it  desirable  to  code  some  of  the  information  via  box  shapes  as  well  as  in 
special  fields.  The  key  to  connector  and  box  number  locations  in  Table  A-l 
will  aid  in  following  the  flow  and  in  finding  boxes  referenced  in  the  tables 
of  Appendix  B. 

The  initial  Simulator  (Model  0)  is  planned  to  operate  at  the  LoSim  level, 
but  will  not  include  Engineering  Change  Proposal  (ECP)  processing,  shown  on 
Sheet  12  of  Figure  A-2.  The  diagram  at  this  level  is  more  systematic  and 
somewhat  less  detailed  than  that  shown  as  mid-level  in  our  April  1979  interim 
report.  While  the  LoSim  level  appears  about  right  to  represent  lifelike 
process  behavior,  level  adjustments,  up  or  down,  can  be  expected  as  the 
Simulator  matures  and  begins  to  support  the  needs  of  its  users.  As  shown  in 
Figure  4,  Model  Box  &  Predecessor/Successor  Counts,  a  total  of  187  boxes  are 


used  in  this  representation  (not  including  the  ECP  Process);  the  figure  also 
provides  a  numeric  breakdown  of  box  types,  Integration  Group  assignments,  etc. 


Note  that  the  complexity  of  the  Model,  in  terms  of  the  number  of  boxes 
and  the  number  of  interconnections,  is  not  critical  at  this  time.  The 
Simulator  design,  described  in  Section  6,  can  readily  support  any  reasonable 
degree  of  complexity.  Higher  complexity  will  add  to  the  time  and  cost  of 
simulation,  and  to  the  effort  needed  to  establish  the  parameter  values  by 
which  each  process  is  quantified;  it  will  not  change  the  Simulator  program, 
however,  which  is  table-driven. 

4.2.3  Expanded  Views  and  Amplification  Notes 

The  LoSim  level  attempts  to  show  valid  lifelike  behavior  of  the 
acquisition  process  while  maintaining  a  manageable  level  of  complexity.  At 
its  level,  however,  some  of  the  activities  portrayed  may  not  be  clearly  enough 
described  or  differentiated.  For  this  reason,  additional  explanatory  material 
is  provided  in  Appendix  A.  The  Process  Flow  Expansion  diagrams  (Figure  A-3) 
further  subdivide  selected  LoSim  box  activities.  The  Process  Flow  Expansion 
diagrams  depict  more  elemental  and  thus  more  easily  understood  activities; 
they  also  clarify  the  box-to-box  flow.  This  material  also  provides  a  better 
functional  basis  for  establishing  parameter  values  to  be  applied  to  the  parent 
LoSim  boxes . 

Note  that  any  Process  Flow  Expansion  can  replace  the  equivalent  LoSim 
box(es)  in  the  Simulator  input  if  the  user  wishes  to  explore  certain  aspects 
of  process  behavior  at  a  lower  level. 

Appendix  A  also  includes  Amplification  Notes  that  clarify  certain 
activities  or  that  provide  relevant  background  material.  These  cover  only  a 
fraction  of  the  total  Model;  their  completion  has  been  deferred  to  FY  80. 


4.3  PRIOR  PROJECT  EXPERIENCE  BASE 

The  definition  of  the  Software  Acquisition  Process  as  described  herein  is 
based  mainly  on  experience  obtained  by  the  authors  through  work  on  prior 
software-related  projects;  the  principal  ones  are  listed  below. 

a.  Project  407L,  Tactical  Air  Control  System  (TACS). 

b.  Project  485L,  Tactical  Air  Control  System  Improvements. 

c.  Project  411L,  E3-A  (formerly  AWACS). 

d.  Project  427M,  Norad  Cheyenne  Mountain  Complex  Improvement  Program. 

e.  SALTY  NET  III.  This  added  a  new  capability  to  the  existing  TACS. 

This  made  it  possible  for  the  TACS  to  achieve  physical  and  operational 
interoperability  with  NATO  Units  via  the  Link  1  Communication  channels. 
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Figure  1.  Major  System  Acquisition  Process 
Normal  Phases  of  Progression 


Figure  2.  Software  Acquisition  Process  Model  Activity  Flow 
Overview  -  Full-Scale  Development  Phase 


MANAGE  &  f  BEGIN  f  A  I  FORM  &  MAN  1  JT PREPARE  MGMT|  p  PROVIDE  DEV  |  JT  PROVIDE  TEST 

SUPPORT  V.  FSD  V  J  n  PROJ  officeI  n  PLANS  J  V"*!  SPRT  FACIL  I  ^  *]  SPRT  facil 


Figure  3.  Software  Acquisition  Process  Flow  Diagram-.-  HiSim  Level 


Figure  4.  Model  Box  &  Predecessor/Successor  Counts 


Development  Example 


SECTION  5 

PROCESS  QUANTIFICATION 


The  Process  Flow  Diagrams  and  Amplification  Notes  discussed  in  Section  4 
describe  the  sequences  of  activities  and  decisions  involved  in  the  acquisition 
(including  development)  of  embedded  software.  Since  this  description  is 
qualitative,  it  can  yield  no  quantitative  predictive  output.  In  this  section 
means  are  described  for  adding  quantity  and  probability  to  the  Process  Model. 

Ultimately,  the  parameters  used  and  their  relationships  will  be  expressed 
generically  so  they  can  apply  to  a  wide  range  of  projects.  That  task  is  most 
difficult  and  will  require  data  sources  with  much  better  definition  and 
control  than  are  now  available.  For  example,  the  planned  ESD  Software 
Acquisition  Resource  Expenditure  (SARE)  reporting  system  will  yield  such  data. 
For  this  year's  effort,  quantities  have  been  developed  to  reflect  those 
obtained  on  a  typical  Air  Force  system.  This  approach  involved  the  steps 
listed  below  and  described  in  the  following  paragraphs : 

a.  definition  of  a  typical  system's  software; 

b.  selection  of  parameters  to  define  each  type  of  functional  element 
contained  in  the  Model; 

c.  estimation  of  parameter  values  for  each  Model  element; 

d.  calibration  and  refinement  of  the  estimated  data. 


5.1  SOFTWARE  DEFINITION  FOR  A  TYPICAL  SYSTEM 

The  size  of  a  typical  project's  software  generally  falls  between  one 
hundred  thousand  and  one  million  lines  of  code.  For  our  work,  an  earlier 
project  (with  which  the  first  author  was  familiar)  was  selected  as  a  mid-range 
example.  This  project,  407L  -  Tactical  Air  Control  System,  was  first 
developed  about  ten  years  ago  (1967-1971)  and  is  still  being  used.  The  data 
used  herein  do  not  correspond  exactly  with  those  of  the  original  development. 
Instead,  they  have  been  modified  to  reflect  changes  in  the  evolving 
development  process,  including  improvements  in  tools  and  techniques  as  well  as 
increased  training  and  deeper  skills.  In  addition,  all  computer  program 
sizes,  time  durations,  and  manpower  levels  have  been  smoothed  and  rounded. 

Based  on  the  above,  the  CPCI  composition  of  a  typical  system  was  defined, 
as  shown  in  Figure  6,  Typical  Project  -  CPCI  Breakdown.  The  most  significant 
and  generally  most  critical  of  these  CPCIs,  (i.e.,  the  real-time  system 
Operating  Program)  was  used  to  estimate  the  typical  CPCI-level  data  for  use 
with  the  Model. 
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A  related  milestone  schedule  is  shown  in  Figure  7,  Typical  Milestone 
Schedule  for  Operating  Program.  Note  that  while  407L  actually  consumed  42 
months,  the  30  month  schedule  in  Figure  7  reflects  our  assessment  of  the 
industry's  current  capability.  Similarly,  the  corresponding  contractor 
manning  profile,  shown  in  Figure  8,  Typical  Manpower  Utilization  Levels,  is 
considerably  lower  than  that  expended  during  the  actual  project.  Finally, 
although  the  original  CPCI  was  not  divided  into  Developmental  Integration 
Groups  (DIGs),  our  Model  assumes  a  three-DIG  division. 

A  set  of  manpower  cost  figures,  based  on  projected  manpower  usage  and 
reflective  of  current  labor  rates,  is  shown  in  Figure  9,  Typical  Manpower 
Costs  for  Operating  Program  CPCI.  These  figures,  which  should  be  viewed  as 
rough  approximations,  are  adequate  to  support  their  purpose  in  this  study, 
which  is  to  provide  a  basis  for  the  calibration  and  refinement  of  the  initial 
parameter  value  estimates.  Consequently,  they  ignore  the  time  value  of  money, 
which  later  Model  versions  will  provide  for.  Also,  Figures  8  &  9  combine  (as 
SYS/TEST)  systems  engineers,  analysts  and  test  engineers,  because  often  the 
same  personnel  perform  these  functions  at  different  times  during  an 
acquisition  program,  and  thus  earn  at  comparable  rates. 


5.2  PARAMETER  SELECTION 

For  each  of  the  three  basic  box  types  represented  in  the  LoSim  and 
Process  Flow  Expansion  diagrams,  the  following  types  of  parameters  have  been 
selected. 

5.2.1  Activity  Box  Parameters 

Each  activity  consumes  resources,  including  manpower  and  time,  often 
expressed  as  man-days.  However,  since  the  number  of  man-days  is  strongly 
influenced  by  the  manning  level,  it  was  decided  that  both  an  appropriate 
manning  level  and  a  duration  would  be  assigned  to  each  activity.  We  plan 
during  FY  80  to  estimate  other  needed  resources,  such  as  computer  time,  but 
have  so  far  concentrated  on  manpower  because  it  is  normally  the  resource  of 
overriding  importance. 

Note  that  the  initial  Model  version  represents  the  manning  levels  and 
durations  independently;  however,  their  estimates  were  developed  jointly. 
Ultimately  the  Model  will  relate  these  and  other  resources  explicitly,  through 
parametric  equations,  which  will  also  reflect  explicitly  the  effects  of 
software  size  and  difficulty,  development  aids,  management  policies,  etc. 

A  number  of  persons  typically  works  on  several  concurrent  activities  on  a 
split  time  basis.  The  Model  allows,  therefore,  for  fractional  manning  levels. 

In  addition,  any  activity  may  need  to  be  repeated  (possibly  several 
times).  Thus,  iteration  factors  are  provided  to  change  the  duration  and 
manning  levels  appropriately  for  each  successive  iteration. 


5.2.2  Decision  Box  Parameters 

During  the  early  development  of  the  process  flow  logic,  no  restrictions 
were  placed  on  the  number  of  outcomes  per  Decision  Box.  As  the  logic  matured 
it  was  observed  that  only  a  few  decisions  had  more  than  two  alternative  exits. 
Since  a  uniform  dual  exit  structure  would  simplify  the  design  and 
implementation  of  the  Simulator,  and  since  dual  exit  boxes  could  be  readily 
staged  to  reflect  multiple  decision  results,  Decision  Boxes  were  restricted  to 
two  mutually  exclusive  exits.  All  decision  questions  were  then  phrased  to 
allow  answers  to  be  uniformly  expressed  as  "Yes"  or  "No".  On  this  basis  the 
probability  of  taking  the  "Yes"  exit  (pYES)  was  assigned  to  each  Decision  Box. 
(Since  pNO  =  1  -  pYES  it  was  unnecessary  to  assign  "No"  exit  probabilities.) 
Since  iteration  can  cause  multiple  reentry  into  a  Decision  Box,  a 
corresponding  set  of  decision  probabilities  was  assigned  to  each  iteration. 

The  assigned  probabilities  are  typical  of  those  observed  on  military 
procurements.  Experience  can  vary  widely,  however,  depending  on  factors  such 
as  contractor  skill  and  experience,  schedule  and  cash- flow  pressure, 
government  monitoring  zeal,  and  urgency  of  need  for  the  software  or  the 
system.  Future  Model  versions  will  allow  the  probabilities  to  vary  with  the 
quantity  and  quality  of  the  effort  expended  on  the  products  on  which  the 
decision  is  based,  as  tempered  by  the  expected  contractual  environment. 

5.2.3  Special  Event  Box  Parameters 

The  Special  Event  Box  supports  two  different  functions.  One  of  these  is 
to  provide  a  label  for  important  events,  usually  called  Milestones,  because  a 
Milestone  may  not  correspond  exactly  with  any  Activity  Box  or  Decision  Box. 

The  other  Special  Event  Box  function  is  to  effect  action  at  another  point 
in  the  process  (e.g. ,  to  increase  the  ECP  frequency  after  each  specification 
is  delivered).  While  the  Special  Event  Box  is  little  used  in  the  initial 
Process  Model  version,  future  versions  will  use  it  extensively  to  allow  the 
computed  quality  of  earlier  activities  to  impact  subsequent  activities  or 
decisions  that  depend  upon  the  earlier  work. 


5.3  PARAMETER  VALUE  ESTIMATION 

Specific  estimates  of  typical  parameter  values  were  required  for  each 
Activity  Box  and  Decision  Box  in  the  LoSim  Process  Flow  Diagram.  The  Activity 
Box  estimates  were  made  by  evaluating,  based  on  personal  experience,  the  work 
that  must  be  accomplished  per  box  and  a  reasonable  manning  level  for  its 
accomplishment.  The  Decision  Box  outcome  probabilities  were  estimated 
similarly.  These  approximations  need  not  be  very  accurate  during  early 
exercise  of  the  Model.  They  can  be  improved  by  the  calibration  process 
described  below.  Appendix  B  describes  and  contains  the  parameter  estimates 
made  to  date. 
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5.4  PARAMETER  VALUE  CALIBRATION 


Since  no  objective  (i.e.,  measured)  data  are  known  to  exist  for  most  unit 
tasks  (activities)  and  decisions  defined  in  the  Model,  the  initial  parameter 
values  were  obtained  as  estimates  from  persons  familiar  with  the  overall 
process.  While  such  values  are  suitable  for  Simulator  development,  the 
validity  of  the  results  obtained  during  its  initial  use  will  depend  on  the 
accuracy  of  the  parameter  values  assigned.  Even  though  Model  uses  such  as 
sensitivity  analyses  can  be  helpful  with  roughly  approximate  data,  the 
validity  of  and  confidence  in  such  results  will  be  higher  if  the  Model 
produces  data  which  are  consistent  with  normal  experience. 

Even  though  no  objective  data  are  available  for  most  unit  tasks,  some 
cost  and  schedule  data  are  available  for  the  overall  software  acquisition 
process.  While  these  data  vary  greatly,  typical  data  can  be  chosen  that  are 
well  within  the  range  of  common  experience;  the  data  described  above  are  an 
example.  The  calibration  process  described  below  uses  typical  overall  cost 
and  schedule  data  to  improve  the  initial  estimates. 

5.4.1  Activity  Box  Timing  Calibration 

Calibration  begins  by  bringing  activity  timing  into  conformance  with  a 
typical  project  schedule,  such  as  that  shown  in  Figure  7,  as  follows: 

a.  The  original  time  estimates  for  the  unit  activities  are  summed  in 
conformance  with  the  sequences  shown  in  the  LoSim  Process  Flow  Diagram 
(Figure  A-2).  This  requires  that  the  Activity  Box  timing  data  and  Decision 
Box  probability  values  be  taken  into  account  with  both  reflecting  the  effects 
of  iteration.  The  details  of  this  process  are  shown  in  Figure  D-l,  Activity 
Duration  Calibration,  and  the  data  obtained  are  summarized  in  Figure  D-2, 
Activity  Duration  Calibration  Summary. 

b.  Using  the  data  obtained  in  Step  a,  a  most  probable  schedule  is 
derived  showing  the  principal  acquisition  Milestones. 

c.  The  milestone  timing  obtained  in  Step  b  is  compared  to  these  same 
Milestones  shown  in  a  typical  acquisition  program  schedule,  derived  as  in 
Section  5.1. 

d.  A  correction  factor  is  then  calculated  for  each  inter-milestone  time 
period.  Each  correction  factor  is  then  applied  against  all  the  Activity  Box 
timing  values  within  the  corresponding  inter-milestone  period. 

This  process  was  applied  to  the  main  (critical)  path  of  the  overall 
process,  covering  the  period  from  contract  award  through  Formal  Qualification 
Testing  (FQT).  The  results,  summarized  in  Figure  10,  Activity  Duration 
Calibration  Summary  -  Main  Path,  show  the  timing  after  the  calibration 
correction  factors  were  applied,  and  after  the  average  effect  of  iteration  had 
been  included.  Figure  D-2  gives  the  same  summary  data  but  shows  the  original 
estimated  data  with  iteration  and,  also,  the  calibrated  data  without 
iteration.  By  comparing  the  figures  it  can  be  determined  that  multiplication 
by  a  factor  of  0.75  was  needed  to  bring  the  original  estimates  into 


conformance  with  the  typical  schedule.  The  results  also  show  that  iteration 
increased  the  overall  acquisition  process  duration  by  an  average  of  28%. 

5.4.2  Manpower  Assignment  Calibration 

Once  timing  values  are  established  for  the  main  path  activities,  and  the 
effect  of  iteration  is  taken  into  account,  the  estimated  manpower  assignments 
to  each  Activity  Box  can  be  calibrated  as  follows: 

a.  An  overall  timing  diagram  is  drawn,  as  shown  in  Figure  11,  Program 
Acquisition:  Activity  Sequence  and  Timing  Diagram.  Every  activity  is  then 
placed  on  the  diagram  at  a  location  conforming  with  its  predecessors'  iterated 
duration.  This  diagram  thus  shows  what  activities  are  likely  to  be  going  on 
at  any  time  during  the  project. 

b.  When  the  manpower  levels  for  each  activity  are  each  assigned  to  its 
appropriate  time  period,  it  becomes  possible  to  sum  the  manpower  level  at  any 
point  in  time. 

c.  When  these  summed  manpower  assignments  are  plotted  vs.  time,  a 
manpower  need  profile  is  created  for  each  category  of  personnel. 

d.  The  manpower  needs  can  then  be  compared  with  the  typical  manpower 
utilization  levels  shown  in  Figure  8. 

e.  The  manpower  levels  assigned  to  each  Activity  Box  can  then  be 
adjusted  (and  smoothed)  for  approximate  conformance  with  the  levels  expected 
on  the  typical  project. 

The  timing  layout  shown  in  Figure  11  preserves  the  functional  and 
connective  relationships  among  the  activities  developed  on  the  LoSim  Process 
Flow  Diagram  (Figure  A-2) .  To  facilitate  reference  to  Figure  A-2,  the  same 
connector  identifications  were  used  in  Figure  11.  Further  work  on  the 
manpower  assignment  level  calibration  was  deferred  to  FY  80,  because  Simulator 
design  was  deemed  more  urgent. 


5 . 5  FUTURE  ACTIONS 

Several  types  of  action  will  be  taken  in  FY  80  and  later  to  improve  the 
accuracy  of  the  Process  Model  parameter  estimates,  and  to  relate  these 
parameters  explicitly  to  one  another  and  to  other  factors  believed  to  affect 
the  elapsed  time  and  costs  of  embedded  software  acquisition.  The  most 
promising  of  these  types  of  action  are  listed  below,  roughly  in  order  of 
increasing  difficulty  and  remoteness.  Each  is  then  briefly  discussed. 

a.  review  by  a  panel  of  experts; 

b.  training  course  use  and  follow-up; 

c.  sensitivity  analysis  using  simulation;  and 


d.  analysis  of  formally  collected  data. 


5.5.1  Review  by  a  Panel  of  Experts 

The  parameter  values  currently  established  represent  the  views  of  the 
authors.  While  we  have  had  considerable  experience  with  military  project 
monitoring,  as  well  as  computer  program  development,  the  current  data  are  more 
reflective  of  subjective  evaluation  and  remembrance  than  of  objective  record. 
Such  subjectively  derived  data  can  be  improved  by  including  the  opinions  of 
many  experienced  personnel.  While  such  a  panel  would  provide  a  basis  for 
consensus  values,  it  also  would  very  likely  show  significant  differences; 
these  latter  point  to  areas  where  further  work  should  be  concentrated.  While 
the  initial  estimates  should  involve  MITRE/ESD  personnel,  contractor  employees 
with  experience  in  military  system  development  would  also  be  valuable 
reviewers.  These  latter  persons  would  be  particularly  qualified  to  provide 
estimates  for  activities  normally  performed  by  contractors. 

5.5.2  Training  Course  Use  and  Follow-Up 

One  of  the  anticipated  uses  of  the  Process  Model  is  for  training  military 
(and  support)  personnel  for  work  on  system  acquisition.  Such  training  would 
include  instruction  on  the  use  of  the  Model  for  project  forecasting  and 
project  monitoring.  During  the  initial  training  classes,  it  will  be  necessary 
to  stress  the  tentative  nature  of  the  initially  used  parameter  values  and  the 
consequent  need  for  tuning  against  real  contract  data.  After  a  cadre  of 
trained  persons  become  involved  in  system  acquisition,  their  reports  on  time 
and  cost  inconsistencies  between  the  actual  and  forecast  figures  will  become 
available.  Cognizant  personnel  at  the  Model  improvement  and  maintenance 
facility  should  use  these  reports  as  a  basis  for  adjusting  the  parameter 
values  (and  cost  estimating  relationships)  to  improve  the  fit  between  Model 
data  and  actual  results. 

5.5.3  Sensitivity  Analysis 

The  parameter  value  calibration  procedure  described  in  Section  5.4  was 
slow  and  arduous  because  it  required  manual  tracing  through  the  complex 
network,  including  Integration  Groups  and  iteration  loops,  while  time  and 
manpower  values  were  calculated  and  summed.  Once  the  Simulator  becomes 
available,  the  network  tracing  and  computation  will  become  automatic,  fast, 
and  accurate.  This  will  allow  the  effects  of  parameter  value  changes  to  be 
reflected  immediately  into  consequent  results.  By  using  this  sensitivity 
analysis  technique  in  this  manner,  the  network  can  be  tuned  quickly  to  reflect 
actual  experience  on  past  (as  well  as  current)  projects. 

In  particular,  when  the  typical  project  estimates  used  in  the  current 
Model  are  replaced  by  generic  functions,  sensitivity  tests  can  support  the 
verification  of  these  functions  and  the  tuning  of  the  associated  parameter 
values.  This  generic  tuning  and  verification  procedure  is  too  complex  to  be 
performed  manually.  For  this  reason,  quantification  refinement  by  this  method 
must  be  deferred  until  the  automated  Model  becomes  usable. 


Once  the  Model  comes  into  use  as  a  contract  monitoring  aid,  feedback  data 
on  the  results  of  its  use  will  allow  the  quantification  of  functions  and 
parameters  to  be  periodically  improved  and  refined.  At  the  same  time,  use  of 
the  Model  will  provide  an  early  indication  of  trends  in  the  quality  of  system 
acquisition  on  an  industry  wide  basis.  It  will  also  provide  a  means  for 
objectivly  comparing  the  quality  of  performance  of  different  contractors,  even 
on  different  projects.  When  this  point  is  reached,  meaningful  performance 
incentives  can  be  added  to  acquisition  contracts. 

5.5.4  Analysis  of  Formally  Collected  Data 

A  separate  ESD/MITRE  activity,  the  SARE  reporting  project,  is  developing 
requirements  for  the  uniform  reporting  of  contractual  performance  data.  When 
SARE  is  applied  to  future  acquisition  projects,  the  data  collected  can  provide 
a  basis  for  establishing  the  quantitative  relationships  needed  for  the  Model. 
Because  of  the  close  relationship  between  the  SARE  project  and  the  Process 
Model,  compatible  cost  relationship  structures  are  being  planned. 
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Figure  6.  Typical  Project  -  CPCI  Breakdown 


Figure  7*  Typical  Milestone  Schedule  For  Operating  Program 


Figure  3  .  Typical  Manpower  Utilization  Levels 
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Figure  9  .  Typical  Manpower  Costs  For  Operating  Program  CPCI 
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Figure  10.  Activity  Duration  Calibration  Summary  -  Main  Path 
(Calibrated  Activity  Durations,  With  Iteration) 


Figure  11.  Program  Acquisition:  Activity  Sequence  and  Timing  Diagram 
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Figure  11.  Program  Acquisition:  Activity  Sequence  and  Timing  Diagram 
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Figure  11.  Program  Acquisition:  Activity  Sequence  and  Timing  Diagram 
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Figure  11.  Program  Acquisition:  Activity  Sequence  and  Timing  Diagram 
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DIAGRAM  NOTATION 


1 .  General : 


Abbissa  shows  elapsed  time  (workdays)  from  start  of  contract. 
Critical  path  is  shown  on  the  uppermost  path. 

The  diagram  was  drawn  to  show  three  DIGs  and  one  TIG. 


2.  Drawing  Conventions: 


Dot 

Arrowhead 
Solid  Line 


•  is  the  start  of  a  box. 

-*•  is  the  end  of  a  box. 

-  is  the  duration  of  a  box. 


Dashed  Lines 


show  paths  to  activities 
not  depicted  on  this 
abridged  diagram 


Dashed  Dotted  Lines 


indicate  slack  time  in 
the  dependency  sequences 


Connectors 


©n  r© 


©H  L® 


Incoming  connectors 
(A&B)  extend  to  the 
left 

Outgoing  connectors 
(C&D)  extend  to  the 
right 


3.  Labeling  Conventions: 


Box  numbers  are  shown  on  or  near  the  solid  line;  subscripts 
denote  group  number. 

Lettered  connectors  correspond  with  those  shown  on  the  LoSim 
Process  Flow  Diagram,  Figure  A-2.  Interior  subscripts 
denote  group  number;  exterior  numerals  denote  the  destina¬ 
tion  page. 

Numbered  connectors  apply  just  to  this  drawing.  Exterior  numerals 
denote  the  destination  page. 


Figure  11.  Program  Acquisition:  Activity  Sequence  and  Timing  Diagram 
Sheet  7 
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SECTION  6 


PROCESS  SIMULATION 


Except  where  otherwise  stated,  this  section  treats  the  development  and 
characteristics  of  the  initial  Simulator  version,  Model  0.  We  intend  Model  0 
to  be  a  relatively  simple  computer  program  that  proves  feasibility  and  that 
provides  a  sound  basis  for  extension.  We  also  intend  Model  0  to  produce  some 
useful  results;  i.e.,  those  indicated  in  Section  6.6.1,  Model  0  Output 
Reports . 

The  principal  Model  0  reports  will  be  profiles  of  the  manpower  needed  to 
perform  the  network's  activities,  and  a  schedule  of  the  major  activity  groups 
(termed  Subnetworks).  These  reports  will  reflect  logical  precedence 
constraints  but  not  manpower  constraints  because  the  latter,  although 
important,  appear  difficult  to  model  realistically. 

The  main  improvement  planned  for  Model  1  will  be  inclusion  of  one  or  more 
algorithms  to  allocate  scarce  manpower,  and  to  effect  delay  whenever  too 
little  of  the  right  manpower  type  is  available.  Model  1  will  also  produce 
dollar  cost  profiles  and  more  elaborate  schedules.  Models  2  and  3  will  each 
have  additional  important  capabilities.  See  Section  6.6.2,  Future  Model 
Output  Reports. 


6.1  DEVELOPMENT  FACILITY  AND  SIMULATION  PROGRAMMING  LANGUAGE  SELECTION 

We  approached  the  problem  of  selecting  a  development  facility  and  a 
simulation  programming  language  for  initial  Simulator  development  as  follows. 
First,  selection  criteria  appropriate  to  project  needs  were  established. 

Then,  promising  candidates  about  which  information  was  readily  available  were 
evaluated  using  the  criteria,  and  choices  were  made.  We  first  selected  the 
development  facility,  because  we  deemed  it  more  important  than  any  specific 
simulation  programming  language,  provided  the  facility  supported  an  adequate 
language.  This  choice  narrowed  considerably  the  number  of  candidate 
languages . 

6.1.1  Facility  Selection 

Six  factors  were  deemed  essential  to  the  initial  Simulator  development: 

a.  affordability  within  project  budgetary  constraints; 

b.  capacity  adequate  for  efficient  compilation  and  execution; 

c.  system  availability  to  Simulator  development  personnel; 

d.  turnaround  time  for  Simulator  development  runs; 

e.  quality  and  availability  of  help  to  solve  apparent  system  problems; 


f.  availability  of  an  adequate  simulation  programming  language. 


Because  our  budget  had  no  funds  for  commercial  computer  rental  or 
purchase,  we  ruled  out  the  use  of  commercial  time-sharing  services  or  block 
computer  time  rental.  Nor  could  we  purchase  a  computer.  Criterion  a.  thus 
limited  us  to  shared  use  of  an  Air  Force  or  MITRE  computer. 

Previous  experience  has  clearly  shown  that  successful  discrete  event 
simulation  requires  substantial  main  memory  and  central  processor  capacity, 
especially  as  the  complexity  of  the  simulation  program  grows  and  the  frequency 
of  simulation  runs  increases.  For  this  reason  (criterion  b.)  we  restricted 
our  search  to  a  large-scale  computer,  ruling  out  mini-computers,  including  one 
otherwise  attractive  candidate,  a  MITRE  PDP  11/45. 

Criterion  c.  meant  that  project  programming  personnel  must  have  the  right 
to  use  the  computer  at  reasonable  times,  and  that  during  most  of  these  times 
they  should  be  able  (directly  or  remotely)  to  access  the  computer  to  enter 
data,  execute  programs,  and  obtain  output.  As  a  practical  matter,  criteria 

а. ,  b.,  and  c.  jointly  limited  us  to  three  computer  systems:  the  Air  Force 
Geophysics  Lab  CDC  computer,  the  RADC  Honeywell  6180,  and  the  MITRE  Bedford 
IBM  370.  Our  choice  among  these  candidate  computers  was  based  primarily  on 
criteria  c.,  d,  and  e,  since  a  satisfactory  simulation  programming  language 
was  available,  or  could  readily  be  provided,  for  each  candidate. 

We  judged  our  ability  to  use  the  IBM  370  far  superior  to  our  ability  to 
use  the  RADC  Honeywell  6180,  mainly  because  we  had  very  limited  access  to 
suitable  terminals  capable  of  flexible  input  and  the  high-volume  output  that 
simulation  program  development  sometimes  demands.  In  contrast,  we  could  use 
several  conveniently  located  CRT  terminals  directly  connected  to  the  IBM  370, 
and  could  easily  walk  to  its  dispatcher's  desk  to  pick  up  printouts. 

Practical  use  of  the  Geophysics  Lab  computer  would  have  entailed  frequent, 
time-consuming  trips  to  Hanscom  AFB. 

The  MITRE  Bedford  Computer  Center  also  provides  expert  help  in  the  use  of 
the  computer  and  its  system  software.  Because  of  its  proximity,  we  decided 
that  this  help  was  superior,  for  our  project  personnel,  to  that  available 
elsewhere.  For  these  reasons  we  selected  the  MITRE  computer. 

б. 1.2  Simulation  Programming  Language  Selection 

Given  the  choice  of  installation,  our  practical  choice  of  languages 
comprised  PL/1,  PASCAL,  FORTRAN,  GPSS,  GASP  IV,  and  SIMSCRIPT  II. 5.  We  ruled 
out  PL/1  in  part  because  it  has  few  compilers  for  non- IBM  computers.  The 
Simulator  should  ultimately  be  transferrable  among  several  different  computer 
families;  this  would  be  difficult  unless  the  Simulator  were  coded  in  a 
language  supported  for  several  machines. 

PASCAL,  despite  some  attractive  features,  lacks  a  well-tested  compiler  at 
the  MITRE  computer  facility.  This  eliminated  it  from  serious  consideration, 
because  our  budget  and  schedule  precluded  our  risking  serious  support  software 
problems . 
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In  practice  FORTRAN  is  as  nearly  universal  as  any  programming  language  in 
use  today.  However,  we  ruled  it  out,  at  least  for  early  Simulator 
development,  because  (like  PL/1  and  PASCAL)  it  lacks  a  timing  routine  and 
other  specialized  features  helpful  to  simulation.  We  believed  (and  still 
believe)  that  a  higher-level  language  could  reduce  Simulator  development 
effort. 

In  contrast,  GPSS  is  a  high-level  language  designed  expressly  for 
discrete  event  simulation.  However,  it  seems  unable  to  support  easily  the 
kind  of  table-driven  program  design  we  envisioned.  In  addition,  we  have 
previous  experience  indicating  excessive  GPSS  storage  and  execution  time 
requirements  as  a  simulation  program  grows.  GPSS  is  also  limited  mainly  to 
IBM  computers,  which  would  impair  transfer  of  a  GPSS  Simulator.  For  these 
reasons  we  rejected  it. 

GASP  IV  is  really  a  collection  of  FORTRAN  subprograms  (e.g.,  a  central 
timing  routine)  designed  to  simplify  development  of  simulation  computer 
programs  written  in  FORTRAN.  Because  these  routines  appear  quite  helpful,  and 
because  GASP  IV  programs  are  as  easily  transferred  as  FORTRAN  programs  (e.g. , 
only  a  FORTRAN  compiler  is  needed),  we  considered  GASP  quite  seriously.  We 
still  deem  it  a  good  second  choice.  However,  we  selected  SIMSCRIPT  I I. 5  over 
GASP  IV  because  of  the  former's  more  easily  readable  vocabulary  (helpful  for 
self-documentation)  and  better  diagnostics.  Also,  former  SIMSCRIPT  II. 5  users 
at  MITRE  recommended  it  highly  and  could  be  consulted  for  help. 

Our  experience  with  SIMSCRIPT  II. 5  to  date  has  generally  been  favorable. 
We  plan  to  continue  using  SIMSCRIPT  II. 5  unless  severe  problems  emerge.  If 
they  do  we  may  switch  to  another  simulation  programming  language. 


6.2  DESIGN  OVERVIEW 

Simulator  Model  0  will  consist  of  three  basic  parts:  (1)  the  Data  Input 
Processor;  (2)  the  Simulation  Conduct  Processor;  and  (3)  the  Output  Report 
Generator.  The  three  processors  will  execute  sequentially.  The  first  will 
operate  on  input  data  sets  defined  in  Appendix  B  to  produce  part  of  a  data 
base  described  in  Section  6.3.  The  second  (the  Simulation  Conduct  Processor) 
will  be  driven  by  these  data.  As  a  result,  the  Simulation  Conduct  Processor 
will  develop  simulation  statistics  which  it  will  store  in  this  data  base.  The 
Output  Report  Generator  will  then  edit  and  print  these  statistics.  The 
designs  of  the  Data  Input  Processor  and  the  Output  Report  Generator  are 
straightforward.  The  design  of  the  Simulation  Conduct  Processor  is  rather 
unusual  (for  a  simulation  program),  and  thus  worth  brief  discussion  here. 

The  Simulation  Conduct  Processor  will  be  table-driven.  Thus,  the 
complete  Model  will  be  defined  by  a  set  of  table  data  such  as  those  given  in 
Appendix  B.  Each  of  the  two-hundred  plus  boxes  included  in  the  LoSim  Process 
Flow  Diagram  of  the  Full-Scale  Development  Phase,  discussed  in  previous 
sections  of  this  report,  is  described  by  an  entry  in  Table  B-l  and  another 
entry  in  Table  B-2,  Table  B-3,  or  Table  B-4.  After  the  Data  Input  Processor 
has  reformatted  these  tables,  the  Simulation  Conduct  Processor  will  read  the 
data  for  the  first  box,  take  actions  which  depend  on  the  box  type  (e.g., 
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Activity  Box,  Decision  Box)  using  the  assigned  parameter  values  (e.g. , 
activity  duration  or  decision  probability),  and  save  any  data  needed  to 
describe  the  results.  It  will  then  proceed  to  each  of  that  box's  immediate 
successor  boxes;  these  will  be  processed  in  appropriate  sequence  until  all 
boxes  involved  in  that  pass  through  the  network  have  been  accessed.  Since  the 
Simulation  Conduct  Processor's  path  through  the  network  will  be  determined  by 
Monte  Carlo  selection  of  alternative  Decision  Box  exits,  a  different  sequence 
of  box  activation  and  results  will  be  likely  on  each  path.  Thus,  the  program 
must  repeat  many  times  (per  another  input  parameter)  to  obtain  statistically 
significant  results. 

This  design  concept  was  selected  because  of  several  desirable  properties. 

a.  The  Simulator's  description  of  the  process  is  easily  changed; 
therefore,  the  nuances  of  each  acquisition  program,  new  facts,  revised 
assumptions,  alternate  policies,  etc.,  can  be  modeled  readily  by  appropriate 
changes  in  the  Simulator ' s  inputs . 

b.  The  computer  program  is  small  and  straightforward,  because  only  a  few 
general  routines  are  needed  to  interpret  the  tables. 

c.  The  program  so  written  can  directly  accommodate  Process  Flow  Diagrams 
and  parameter  values  developed  for  other  acquisition  life  cycle  phases,  or 
even  for  different  processes.  For  example,  it  could  readily  simulate: 

(1)  the  other  aspects  (e.g.,  the  hardware-related  and  personnel- 
related  activities)  of  Major  System  acquisition  programs; 

(2)  programs  managed  per  AFR  300-series  regulations; 

(3)  acquisitions  conducted  per  Army,  Navy  or  other  agency 
regulations ; 

(A)  other  processes  that  principally  involve  groups  of  people 
working  toward  common  goals. 

As  a  result,  in  contrast  to  many  simulation  programs,  which  have  a  block  of 
code  for  each  different  section  of  process  flow,  the  Simulator  promises  to  be 
relatively  simple  to  build,  easy  to  adapt  or  extend,  and  inexpensive  to 
maintain. 


6.3  DATA  BASE 

A  central  data  base  referenced  by  the  three  processors  is  essential  to 
this  design  concept.  The  part  of  the  data  base  that  defines  the  process 
network,  the  process  parameters,  and  simulation  options,  will  be  created  by 
the  Data  Input  Processor  from  inputs  defined  in  Appendix  B  and  illustrated  in 
Tables  B-l  through  B-4.  The  Data  Input  Processor's  primary  function  is  to 
transform  this  input  into  a  structure  that  the  Simulation  Conduct  Processor 
(which  will  execute  many  times  per  simulation  run)  will  process  efficiently. 
The  structured  input  data  will  then  guide  the  Simulation  Conduct  Processor 
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through  the  network,  developing  and  storing  the  data  needed  by  the  Output 
Report  Generator. 

The  structure  and  content  of  the  data  base  so  far  developed  for  Simulator 
Model  0  is  shown  in  Figure  12,  Data  Base  Structure.  Extracts  from  the 
compiled  version  are  included  within  Appendix  E,  Simulation  Program  Listing 
Extracts.  The  figure  shows  a  Main  Entry  for  each  box  of  the  Model.  This 
contains  information  of  general  utility  that  occupies  a  fixed  amount  of 
storage.  Each  Main  Entry  also  contains  pointers  to  as  many  as  five  lists 
(called  Sets  in  SIMSCRIPT  II. 5)  which  contain  the  remaining  data  for  the  box, 
as  explained  below. 

6.3.1  List  Structure 


The  data  base  structure  relies  on  the  list-handling  capabilities  of 
SIMSCRIPT  II. 5.  As  described  below,  list  structures  help  to  deal  with  the 
complex  data  that  characterize  the  Process  Model  design.  In  Model  0  the 
following  lists  may  belong  to  each  box's  Main  Entry: 

a.  A  Pred  lists  all  boxes  that  are  immediate  predecessors  of  the  owning 
box.  It  also  defines  conditions  for  starting  that  box,  for  iterating  that 
box,  or  for  executing  that  box  for  different  Integration  Groups.  A  Pred  is  a 
list  because  a  box  may  have  a  variable  number  of  predecessors. 

b.  A  Yes.Succ  lists  all  boxes  that  immediately  follow  a  Decision  Box's 
"Yes"  exit.  A  Yes.Succ  also  lists  the  immediate  successors  of  an  Activity  Box 
or  a  Special  Event  Box. 

c.  A  No.Succ  lists  all  boxes  that  immediately  follow  a  Decision  Box's 
"No"  exit.  A  No.Succ  for  a  Special  Event  may  point  to  other  boxes  which  are 
the  objects  of  parameter  changes  or  reset  actions.  The  No.Succ  pointers  of 
Activity  Boxes  are  empty. 

d.  An  OTD  (Occurrence  and  Timing  Data)  lists  certain  data  (e.g. , 
predecessor  completion  data,  start  and  finish  times)  that  may  vary  with  DIG  or 
TIG.  There  is  one  entry  in  the  list  for  each  DIG  or  TIG.  For  boxes  not 
involved  in  DIG-  or  TIG- repetition,  there  is  one  OTD  entry. 

e.  An  ATM  (Activity  Timing  and  Manpower)  lists  the  estimated  manpower 
levels  (by  manpower  type)  and  durations  necessary  to  complete  each  activity. 
There  is  a  separate  ATM  Entry  for  each  DIG  or  TIG,  if  any.  Otherwise  there  is 
a  single  ATM  entry  for  the  activity  as  a  whole.  Only  Activity  Boxes  have 
ATMs.  The  ATM  pointers  of  Decision  Boxes  and  Special  Event  Boxes  are  empty. 

6.3.2  Data  Packing 

Figure  12  shows  that  SIMSCRIPT  II. 5  permits  packing  several  data  fields 
into  the  same  computer  word;  e.g.,  word  two  of  the  Main  Entry  contains  seven 
different  data  fields.  This  significantly  reduces  the  storage  space  needed  to 
contain  the  data  base. 
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6.3.3  Data  Base  Size 


The  data  base  thus  far  defined  in  detail  tor  Simulator  Model  0  at  the 
LoSim  level  comprises  about  11,000  computer  words,  as  shown  in  Figure  13,  Data 
Base  Size.  An  average  of  57  words  of  storage  is  needed  per  box. 

The  data  elements  defined  in  Figure  13  will  store  model  definition 
information  and  some  interim  run  results,  plus  some  overhead  (pointers  and 
counters).  The  latter  are  needed  to  preserve  the  relationships  among  the 
separately  stored  segments  of  data.  By  counting  the  overhead  words,  the 
overhead  was  found  to  account  for  about  20%  of  the  total. 

In  addition,  the  system  will  require  storage  for  system  and  local  data 
plus  storage  for  data  extracted  for  statistical  analysis.  These  storage  needs 
have  not  yet  been  defined  in  detail.  Thus,  they  cannot  yet  be  quantitatively 
analyzed,  but  are  believed  reasonable. 


6.4  DATA  INPUT  PROCESSOR 

The  Data  Input  Processor  will  read  in  the  designated  Model  definition 
data  (per  Appendix  B),  and  (in  later  Simulator  versions)  data  that  describe 
the  system  environment  and  output  options.  It  will  check  this  input  for 
format,  consistency,  and  completeness.  If  the  data  do  not  meet  all  acceptance 
criteria,  the  program  will  output  appropriate  diagnostic  messages;  otherwise 
the  data  will  be  restructured  and  loaded  into  the  data  base. 

The  Data  Input  Processor  will  then  initialize  the  Simulator  to  begin  the 
specified  number  of  passes  through  the  network,  and  will  then  transfer  control 
to  the  Simulation  Conduct  Processor.  A  compiler  listing  that  includes  the 
Data  Input  Processor  code  written  to  date  is  included  within  Appendix  E. 


6.5  SIMULATION  CONDUCT  PROCESSOR 

This  program  will  traverse  the  network  from  the  initial  element  (e.g., 

Box  2A)  through  the  final  elements  (e.g.,  Boxes  82Y  and  54V).  It  will 
simulate  each  activity  and  decision  (and  will  perform  every  Special  Event), 
along  paths  selected  randomly  per  decision  probabilities.  While  following  the 
network,  it  will  develop  statistics  on  decision  outcomes,  delays,  demand  for 
resources,  resource  utilization,  iteration,  etc. 

The  Simulation  Conduct  Processor  will  repeat  its  network  traversal  as 
many  times  as  specified  in  an  input  parameter  termed  N. Repetitions .  During 
each  traversal  it  may  follow  a  somewhat  different  path,  because  different 
decision  outcomes  will  be  selected  randomly.  The  Event  Notice  technique, 
provided  by  the  SIMSCRIPT  II. 5  language,  will  control  the  Simulation  Conduct 
Processor  as  described  below. 


6.5.1  Event  Notice  Concept 


a.  A  SIMSCRIPT  II. 5  Event  Notice  initiates  a  defined  set  of  activities 
at  a  scheduled  simulated  time,  specified  in  each  Event  Notice.  The 
SIMSCRIPT  II. 5  Schedule  verb  creates  each  Event  Notice  and  files  it  in  an 
appropriate  list. 

b.  The  designer  can  define  any  number  of  different  Event  Types.  For 
each  Event  Type  there  is  a  defined  set  of  parameters,  and  an  associated 
computer  routine  (termed  an  Event  Routine)  which  accomplishes  the  functions 
specified  for  that  Event  Type.  These  functions  may  include  scheduling 
additional  Events . 

c.  SIMSCRIPT  II. 5  maintains  lists  (one  for  each  Event  Type)  which  hold 
Event  Notices  waiting  to  be  processed.  Whenever  an  Event  Notice  has  been 
processed  it  is  normally  destroyed,  and  control  returns  to  the  SIMSCRIPT  II. 5 
timing  routine  which  searches  all  the  Event  Lists  to  find  the  Event  Notice 
with  the  now  earliest  scheduled  time.  Special  priority  logic  resolves  ties. 
The  timing  routine  then  actuates  the  earliest  Event  and  sets  Simulation  Time 
to  that  Event's  time. 

d.  Every  SIMSCRIPT  II. 5  program  includes  a  Main  Program.  Among  other 
things  this  must  schedule  at  least  the  first  Event  and  start  simulation;  i.e., 
transfer  control  to  the  timing  routine  to  select  the  first  Event  Notice  to  be 
processed.  When  the  timing  routine  exhausts  all  Event  Notices,  it  returns  to 
the  Main  Program. 

6.5.2  Event  Notice  Use 

Simulator  Model  0  has  been  designed  to  use  two  Event  Types  which  are 
defined  in  Appendix  F,  Event  Notice  Specification.  One  of  these  (Box.Proc) 
performs  the  processing  associated  with  the  particular  box  designated  in  the 
Event  Notice.  This  one  Event  Type  will  be  used  to  process  all  box  types  used 
in  the  Model. 

The  other  Event  Type  (Flow)  will  be  scheduled  each  time  Box.Proc 
completes  its  processing.  Flow  will  perform  any  needed  termination  processing 
for  the  designated  box,  and  then  update  and  examine  the  input  status  for  each 
of  the  boxes  that  immediately  succeed  the  designated  box.  For  each  of  the 
successor  boxes  on  which  all  input  conditions  have  been  satisfied,  a  Box.Proc 
Event  Notice  will  be  scheduled. 

6.5.3  Simulation  Conduct  Operation 

The  Simulation  Conduct  Processor  will  begin  after  the  Data  Input 
Processor  has  loaded  and  reformatted  the  input  data,  and  has  created  and 
scheduled  an  Event  Notice  for  the  first  box  of  the  Model  (e.g.,  Box  2A) .  Then 
the  SIMSCRIPT  II. 5  timing  routine  will  be  called.  When  it  searches  its  Event 
Notice  Lists  it  will  find  just  the  one  Event  Notice  (a  Box.Proc  for  the 
initial  box)  inserted  by  the  Data  Input  Processor.  The  timing  routine  will 
then  set  Simulation  Time  to  the  Event  time  (which  is  zero)  and  call  the 
Box.Proc  Event  Routine.  This  will  begin  a  sequence  of  Event  Notice  processes 


such  as  that  diagrammed  in  Figure  14,  Event  Notice  Processing  Sequence.  This 
figure  shows  how  the  Event  Notices  are  linked  together  to  accomplish  the 
actions  described  in  the  simulation  tables ;  it  provides  an  abridged  view  of 
the  process  up  to  PDR  #1. 

As  shown  in  the  figure  the  first  Box.Proc  (for  Box  2A)  will  accomplish 
the  functions  associated  with  Box  2A  (an  Activity  Box),  and  then  schedule  a 
Flow  Event  to  occur  on  day  10  (because  (per  Table  B-2)  the  activity  duration 
of  Box  2A  is  10  days).  The  Flow  Event  Routine  for  Box  2A  will  terminate  the 
Box  2A  activity  and  then  (per  Table  B-l)  schedule  two  new  Box.Proc  Events  (for 
Boxes  4A  &  4S)  for  iimnediate  action.  The  Box  4A  Box.Proc  Event  can  only 
update  status;  it  needs  another  input  (from  Box  4S)  before  its  activity  can 
begin.  When  the  Box  4S  Event  does  begin  it  will  schedule  a  Box  4S  Flow  Event 
for  15  days  later. 

The  SIMSCRIPT  II. 5  Controller  will  now  find  that  the  Box  4S  Flow  Event  is 
the  earliest  so  it  will  add  15  days  to  Simulation  Time  and  then  invoke  the 
Event.  The  Flow  Event  program  will  now  create  immediate  Box.Proc  Events  for 
the  three  successors  of  Box  4S:  Boxes  4A,  60A,  and  62A.  Box.Proc  processing 
for  both  boxes  60A  &  62A  will  begin  immediately;  for  each  a  Flow  Event  will  be 
scheduled.  A  Box.Proc  Event  for  Box  4A  will  also  begin  immediately  because 
both  of  this  box's  inputs  (Boxes  2A  and  4S)  will  be  satisfied;  it  will 
schedule  a  concluding  Flow  Event  for  Box  4A  10  days  hence.  The  Box  4A  Flow 
Event  Routine  will  then  schedule  immediate  Box.Proc  Events  for  its  successor 
boxes  (4C,  6A,  53A,  and  53C) . 

This  interlaced  Box.Proc  and  Flow  Event  processing  will  continue  until 
there  are  no  more  Event  Notices  to  process.  Several  cases  not  described  above 
deserve  mention.  Whenever  a  Decision  Box  Box.Proc  is  processed  (as  shown  for 
Box  6E)  the  exit  will  be  selected  in  zero  Simulation  Time.  The  diagram  shows 
that  the  "No"  exit  would  be  selected  on  first  pass  (day  72)  so  the  diagram 
shows  the  iterative  return  to  Box  6A.  The  next  time  6E  would  be  reached  (day 
84)  the  "Yes"  exit  to  Box  6G  would  be  shown. 

The  other  feature  involves  the  handling  of  specified  internal  delays; 
e.g.,  on  Box  4E.  In  this  case,  the  Flow  Event  for  Box  4C  (day  47)  will  detect 
the  Box  4E  Wait  and  schedule  the  Box  4E  Box.Proc  to  begin  after  the  (5  day) 
internal  wait  period. 

6.5.4  Simulation  Results  Data  Collection 

During  simulation  conduct,  timing,  resource  utilization,  and  decision 
outcome  data  will  be  collected  within  the  data  base.  A  list  of  all  such  data 
planned  for  Model  0  is  provided  in  Appendix  G,  Demonstration  Model  (MO) 
Statistics  Development.  In  general,  data  will  be  collected  for  individual 
boxes,  for  the  whole  network,  and  for  user-defined  network  subdivisions 
(Subnetworks).  The  data  will  be  collected  primarily  to  support  the  needs  of 
the  Output  Report  Generator  program.  Some  of  the  data  however,  are  planned  to 
aid  test  and  verification. 
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6.5.5  Multi-pass  Control 


Because  all  Decision  Box  processing  and  most  activity  duration 
computation  will  include  probabilistic  elements,  each  pass  through  the  network 
will  yield  somewhat  different  data.  Therefore,  as  described  in  Appendix  G, 
each  simulation  run  will  include  many  passes  through  the  network  to  obtain 
results  with  statistical  significance.  The  multi-pass  control  will  provide 
the  means  for  conducting  the  input-specified  number  of  passes,  extracting  and 
aggregating  the  data  from  these,  and  reinitializing  the  per-pass  data  base 
between  each  pair  of  passes. 


6.6  OUTPUT  REPORT  GENERATION 

When  the  Simulation  Conduct  Processor  has  completed  its  tasks,  many 
passes  through  the  network  will  have  been  completed  and  raw  data  reflecting 
the  simulation  results  will  have  been  accumulated.  The  Output  Report 
Generator  will  reduce  this  data  into  defined  statistical  results,  and  will 
prepare  reports  which  encapsulate  them,  based  on  report  selection  input 
parameters. 

These  output  reports  will  provide  information  about  system  entities  at 
three  levels,  as  follows. 

a.  The  lowest  level  entities  are  individual  function  boxes. 

b.  The  highest  level  entity  is  the  total  network. 

c.  The  intermediate  level  entities  are  subdivisions  of  the  total 
network,  hereafter  termed  Subnetworks .  Each  Subnetwork  consists  of  a  group  of 
function  boxes  which  the  user  may  designate  to  identify  some  larger 
(aggregate)  function.  For  example,  all  the  boxes  shown  on  Sheet  3  of  the 
LoSim  Level  Process  Flow  Diagram  (Figure  A-2)  could  be  formed  into  a 
Subnetwork  which  included  all  the  detailed  design  activities.  The  Model  0 
user  will  be  able  to  define  up  to  15  Subnetworks,  by  marking  each  box  involved 
with  the  number  of  a  single  Subnetwork  to  which  it  belongs. 

Whenever  applicable,  separate  data  values  will  be  accumulated  for  those 
components  of  the  total  which  result  from  process  iteration.  Also,  for 
processes  repeated  for  different  Integration  Groups,  separate  values  will  be 
retained  for  each  Group.  For  most  statistical  data,  the  mean  value,  the 
standard  deviation,  the  number  of  instances,  the  minimum,  and  the  maximum 
value  encountered  will  be  available  for  output. 


The  Model  0  output  reports  are  defined  in  Appendix  G.  In  general,  they 
will  provide  the  information  listed  below  for  each  of  the  following  types  of 
system  entity: 

a.  Activity  Boxes 

(1)  the  time  expended  in  waiting  for  all  predecessors'  input 
conditions  to  be  satisfied; 

(2)  the  total  time  used  for  internal  waiting  (i.e.  starting  delays 
inherent  in  the  activity) ; 

(3)  the  total  time  used  performing  the  activity; 

(4)  the  earliest  start  time  and  latest  finish  time; 

(5)  the  count  of  the  number  of  iterations;  and 

(6)  the  total  of  each  category  of  manpower  used. 

b.  Decision  Boxes 

(1)  the  predecessor  wait  and  internal  wait  times  as  well  as  the 
iteration  count,  as  described  for  item  a.  above; 

(2)  the  earliest  and  latest  occurrence  times;  and 

(3)  a  record  of  the  exit  selections  made. 

c .  Subnetworks 

(1)  the  earliest  start  and  latest  completion  times  (or  equivalent 
occurrence  times);  and 

(2)  the  total  of  each  category  of  manpower  used. 

d.  Full  Network 


(1)  the  total  of  each  category  of  manpower  used; 

(2)  a  profile  of  personnel  use  (i.e.,  manning  level)  for  each 
manpower  category  and  total  manpower;  and 

(3)  a  project  schedule  showing  milestone  times. 

6.6.2  Future  Model  Output  Reports 

The  model  simulation  technique  is  capable  of  producing  much  more 
extensive  information  than  that  shown  for  Model  0.  Examples  of  the  inherent 
possibilities  are  shown  in  Appendix  H,  Data  Reporting  for  Later  Simulator 


Models.  The  growth  in  output  capability  is  planned  to  be  evolutionary.  The 
expected  user  participation  during  the  growth  period  will  help  assure  that  the 
reports  will  contain  the  information  needed  for  the  numerous  applications  seen 
for  the  Simulator. 


MAIN  ENTRY - ,  . -  ASSOCIATED  LISTS  (SETS) 


The  model  network  at  LoSim  level  includes  191  boxes  (not 
including  ECP  processing)  with  the  breakdown  shown  in  Figure  4. 
Data  base  storage  is  based  on  3  DIGS  and  5  TIGS,  as  follows: 


UNIT 

TOTAL 

UNIT  STORAGE 

ELEMENT 

COUNT 

COUNT 

(WORDS) 

TOTAL  WORDS 

BOXES 

191 

191 

9 

1719 

PRED(ECESSORS) 

286 

286 

2 

572 

YES .  SUCCESSORS  ) 

249 

249 

2 

498 

NO.  SUCCESSORS  ) 

57 

57 

3 

171 

OTD( 1-Group) 

113 

113 

OTD( 3-Group) 

51 

153 

OTD(5-Group) 

23 

115 

TOTAL 

191 

401 

4 

1604 

ATM( 1-Group) 

70 

70 

ATM( 3-Group) 

40 

120 

ATM( 5-Group) 

23 

115 

TOTAL 

133 

305 

21 

6405 

TOTAL  BOX  DATA  STORAGE 

10,969 

AVERAGE  STORAGE 

PER  BOX  - 

_  _  _  _ 

-  -  57  WORDS 

Figure  13.  Data  Base  Size 
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Figure  14.  Event  Notice  Processing  Sequence 


SECTION  7 


ACCOMPLISHMENTS  AND  STATUS 


7 . 1  ACCOMPLISHMENTS 

The  work  to  date  on  the  Software  Acquisition  Process  Model  was  conducted 
and  documented  in  three  major  time  segments.  During  the  initial  phase,  the 
software  acquisition  problem  was  defined,  prior  art  solutions  were  evaluated, 
the  Process  Model  concept  was  described,  a  number  of  applications  were 
identified,  and  Model  feasibility  was  explored.  The  results  of  that  work 
indicated  that  the  Model  was  feasible  and  desirable. 

During  the  second  phase  of  the  project,  major  emphasis  shifted  towards 
development  of  a  diagramatic  representation  of  the  acquisition  process  which 
could  support  the  mechanization  of  the  Model.  Process  Flow  Diagrams  in  three 
levels  of  detail  were  developed  to  a  point  whe>re  they  could  be  compared  and 
evaluated.  It  was  concluded  that  the  middle  level  was  best  for  simulation, 
while  the  other  levels  had  supplementary  value.  During  this  period,  the 
initial  Simulator  programming  language  selection  decision  was  made  and  some 
exploratory  coding  was  accomplished.  Finally,  the  parameter  value  estimation 
technique  was  selected  and  the  decision  was  made  to  limit  the  initial  scope  of 
the  effort  to  the  Full-Scale  Development  Phase  of  the  overall  acquisition 
process . 

With  the  preliminary  investigation  and  decisions  completed,  the  final 
portion  of  the  work  to  date  was  concentrated  on  development  of  a  satisfactory 
design  concept  and  initial  implementation  of  the  design.  The  Model  definition 
process  was  refined  and  essentially  completed.  The  quantification  process  was 
extended  to  allow  estimated  parameter  values  to  be  refined  by  calibration. 
During  this  final  period,  a  very  promising  set  of  Simulator  design  concepts 
was  evolved  and  adopted.  Much  of  the  very  rapid  progress  described  in  the 
technical  portions  of  this  document  are  attributable  to  this  design.  As  a 
result,  a  successful  outcome  to  this  project  is  most  likely  and  the  problem 
solving  potential  of  the  Simulator  still  appears  to  be  viable  and  realizable. 


7 . 2  CURRENT  STATUS 

The  current  status  of  the  principal  project  tasks  is  described  below. 

All  the  status  information  applies  to  the  work  on  Simulator  Model  0  and  the 
Full-Scale  Development  Phase  of  the  Major  System  acquisition  process,  which 
has  been  the  focus  of  the  FY  79  effort.  While  quantitative  terms  have  been 
used  to  express  the  degree  of  completion  on  some  tasks,  these  are  necessarily 
approximate.  Also,  some  work  shown  to  be  complete  might  require  revision 
later,  because  the  close  interrelationship  between  the  various  tasks  could 
cause  new  work  to  perturb  previously  completed  work. 
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7.2.1  Process  Model  Definition 

a.  Overview  Diagrams:  completed 

b.  Simulation  Level  (HiSim  and  LoSim)  Diagrams:  completed 

c.  Expanded  View  Diagram:  20%  completed 

d.  Amplification  Notes:  20%  completed 

e.  Network  Linkage  Table:  completed. 

7.2.2  Process  Model  Quantification 

a.  Parameter  Selection:  completed 

b.  Parameter  Value  Estimation: 

(1)  Decision  Probabilities:  completed 

(2)  Activity  Timing:  50%  completed 

(3)  Manning  Levels:  25%  completed. 

c.  Parameter  Calibration: 

(1)  Concept:  complete 

(2)  Timing:  main  threads  (critical  path  only)  completed 

(3)  Manning:  10%  completed. 

d.  Parameter  Tables: 

(1)  Structure  and  Definition:  LoSim  level  complete 

(2)  Data  Content:  all  current  estimates  are  entered. 

7.2.3  Process  Model  Simulation 

a.  Data  Base  Definition:  completed  for  all  box  data 

b.  Data  Base  Coding:  completed  for  all  box  data 

c.  Model  Control  Concepts  and  Structure:  completed 

d.  Model  0  Outputs:  defined 

e.  Future  Model  Outputs:  tentative  definition  prepared 

f.  Event  Notice  Program  Definitions:  completed  for  all  box  data  (except 
for  Special  Event  Boxes) 


g.  Data  Input  Processor  Coding:  mostly  exploratory;  5%  completed. 


SECTION  8 


ANTICIPATED  GROWTH 
AND  PLANS 


8 . 1  GROWTH  AREAS 

As  part  of  the  phased  implementation  approach  being  used  on  this  project, 
Model  0  includes  a  central  core  of  essential  capabilities  with  built-in  growth 
support  features.  Listed  below  are  additional  capabilities  which  are  likely 
candidates  for  inclusion  in  later  Models.  The  growth  areas  have  been  grouped 
into  the  task  areas  (i.e..  Definition,  Quantification,  and  Simulation) 
previously  used  throughout  this  report.  It  is  to  be  understood,  however,  that 
each  new  capability  will  probably  have  some  impact  on  all  three  areas. 

8.1.1  Model  Definition 


a.  The  LoSim  Diagram  (Figure  A-2)  needs  review  by  others  and  refinement. 
It  should  be  simplified  by  combining  some  lesser  functions  into  larger  ones. 

It  should  provide  for  more  flexible  Integration  Grouping;  i.e.,  some  difficult 
tasks  should  be  permitted  to  begin  in  Group  1  and  to  conclude  with  a  later 
group . 

b.  The  FSD  Model  should  be  made  to  represent  interdependencies  among 
CPCIs  and  also  with  any  interfacing  equipment  Configuration  Items  (CIs)  which 
are  being  concurrently  developed  or  procured. 

c.  Interfacing  Models  should  be  created  for  the  other  Major  System 
Acquisition  Life  Cycle  phases,  covering  the  embedded  software  acquisition 
process;  i.e.,  the  Validation,  Conceptual,  Production,  and  Deployment  phases, 
probably  in  that  order. 

d.  The  Modeled  process  should  eventually  cover  the  procurement  of  entire 
Electronic  Systems,  including  the  acquisition  and  development  of  special 
hardware  CIs. 

e.  The  representation  of  the  ECP  process  needs  to  be  improved  so  that 
its  impact  on  and  relationships  with  the  other  activities  and  discussions 
becomes  a  reasonable  reflection  of  actual  practices. 

8.1.2  Quantification 


a.  All  currently  used  numerical  values,  which  reflect  "typical"  usage 
should  be  replaced  by  generic  functional  relationships,  which  can  therefore  be 
applied  to  a  large  variety  of  contractual  and  developmental  situations;  see 
Section  5.5. 

b.  The  generic  relationships  should  be  developed  in  conjunction  with 
these  developed  on  the  SARE  project;  see  Section  5.5.4. 


c.  The  time,  resource,  and  probability  quantities  associated  with  an 
activity  or  decision  should  be  made  to  reflect  the  quality  of  prerequisite 
prior  activities  and  decisions.  While  the  quality  of  an  intermediate  product 
(i.e.,  an  output  that  is  produced  by  an  Activity  Box)  is  not  directly 
assessed,  it  can  be  inferred  in  some  cases  from  the  quantity  and  quality  of 
the  effort  expended.  I.e.,  if  person  (or  group)  with  a  given  capability 
requires  thirty  days  to  properly  accomplish  a  task,  but  actually  uses  fifteen 
days,  the  Model  could  assume  that  the  output  product  is  poor.  Therefore,  any 
decisions  based  on  the  product  should  be  biased  toward  non-acceptance  and  any 
subsequent  actions  which  use  the  product  should  be  biased  towards  longer 
duration  or  poorer  quality.  Similarly,  if  an  acceptance  decision  resulted  in 
iteration  of  the  activity  (i.e.,  rework),  the  output  product  quality  would 
normally  be  improved.  This  quality  dependency,  while  complex  and  difficult  to 
quantify,  is  necessary  if  the  Model  is  ever  to  achieve  a  life-like 
representation  of  the  acquisition  process.  The  draft  Data  Item  Description 
for  SARE  reporting  includes  parameters  that  define  quantitative  measures  of 
quality  (and  other  factors  that  affect  software  time  and  resource 
consumption).  We  expect  that  these,  or  variations  of  them,  can  be  used  in  the 
Model . 

d.  Resource  utilization  parameters  in  addition  to  manpower  should  be 
included.  In  particular,  facilities  used  for  program  development  and  for  test 
must  each  be  quantified  to  reflect  its  maximum  utilization  rate  as  well  as  the 
rate  of  use  by  each  pertinent  Activity  Box.  The  Model  must  include  capability 
to  represent  separate  quantities  of  as  many  kinds  of  resources  as  may  be  used. 

e.  The  number  of  manpower  types  assigned  to  each  task  may  need  to  be 
changed  to  reflect  subdivision  beyond  those  currently  used;  see  Section  4.1.7. 

f.  No  dollar  cost  data  are  developed  in  Model  0.  Dollar  cost  data 
reflecting  manpower  use  should  soon  be  added,  and  that  reflecting  other 
resources  should  follow  shortly.  The  dollar  cost  data  will  also  need  to 
reflect  the  changing  value  of  money  during  the  acquisition  period. 

8.1.3  Simulation 

When  implemented,  all  changes  described  above  (e.g.,  those  regarding 
quality)  will  impact  the  Simulator.  A  few  deserve  special  attention  as 
provided  below. 

a.  The  initial  Model  will  maintain  records  of  manpower  usage,  but  will 
not  limit  the  quantities  available.  Since  the  resources  available  usually 
constrain  real  projects,  later  versions  of  the  Model  will  need  to  retain 
quantitative  data  on  the  availability  limits  for  each  kind  of  resource.  This 
situation  requires  that  the  Model  eventually  reflect  the  following: 

(1)  Manpower  availability  varies  with  time  as  a  result  of  factors  such 
as  planned  build-up  and  reduction,  employee  transfer  or  resignation, 
and  sickness. 


(2)  Other  resource  availability  also  varies  with  time  due  to  equipment 
acquisition  time,  equipment  maintenance  and  unscheduled  down  time, 
contention  with  other  in-house  projects,  etc. 

(3)  On  any  real  project,  the  allocation  of  scarce  resources  among 
competing  activities  is  resolved  by  real-time  management  action.  In 
a  simulation  model,  the  allocation  is  usually  resolved  by  a 
management  stragegy  function.  Since  many  different  management 
strategies  may  be  used  including  complex  "look  ahead"  functions,  the 
development  of  this  area  is  planned  to  proceed  in  an  evolutionary 
manner.  Therefore,  early  versions  will  use  simple,  primitive 
strategies  (e.g.,  wait  until  all  needed  resources  are  available); 
later  ones  will  allocate  and  move  resources  on  the  basis  of 
impending  urgency,  including  the  effects  of  transiency. 

b.  At  various  points  during  a  project,  a  completed  event  can  trigger  a 
multiple  burst  set  of  like  activities.  E.g.,  completion  of  a  CDR  leads 
immediately  to  simultaneous  coding  by  many  programmers  on  many  program 
modules.  The  initial  Model  treats  these  situations  as  randomly  variable 
phenomena,  with  each  burst  having  one  beginning  and  one  end.  On  real 
developments,  the  many  unit  tasks  proceed  With  different  rates  and  with 
differing  degrees  of  success.  The  real  world  can  therefore  show  integration 
problems  (e.g.,  a  critical  module  is  not  ready)  which  are  not  reflected  by  the 
initial  Model.  While  it  is  not  considered  practical  to  model  burst  activities 
at  a  level  which  reflects  each  worker's  accomplishments,  it  will  eventually  be 
necessary  to  find  a  modeling  level  which  does  represent  multiple  bursts 
realistically. 

c.  The  modeling  of  the  development  of  multiple  interdependent  CPCIs  can 
be  handled  by  the  Simulator  Model  0  design.  Each  CPC1  can  be  separately 
treated,  with  each  independently  following  its  normal  process  flow  diagram, 
with  its  own  parameter  values,  function  box  numbers,  and  idiosyncratic 
adjustments.  To  do  this  data  which  define  all  the  CPCIs  will  be  included  in 
the  function  box  definition  tables  defined  in  Appendix  B.  All  precedence 
dependencies  among  the  CPCIs  will  then  be  reflected  by  inclusion  in  the 
Network  Linkage  Table  (Table  B-l);  this  treatment  is  therefore  the  same  as  for 
intra-CPCI  dependencies.  However,  later  Simulator  versions  may  incorporate  a 
way  to  treat  multiple  CPCIs  more  compactly. 

d.  The  simulation  of  ECP  behavior  does  create  special  simulation 
problems,  such  as: 

(1)  ECP  occurrences  on  real  contracts  are  generally  in  response  to 
problem  situations  which  cannot  be  forecast  on  an  individual  basis. 
Certain  periods  during  the  development,  however,  are  more  prone  to 
produce  ECPs.  The  planned  Simulator  design  provides  for  spontaneous 
generation  of  ECPs  at  a  controllable  rate  which  is  randomized  when 
applied. 

(2)  The  generation  of  an  ECP  consumes  intrinsic  effort  which  is  easily 
modeled.  It  can  also  profoundly  effect  the  time  and  costs  of 


development  activities  not  yet  reached  or  may  require  rework  on 
already  completed  activities. 


These  effects  can  be  simulated  in  the  Model  by  the  following  means. 
Special  Event  Boxes  in  the  Model's  ECP  processor  will  alter  the  time  and 
effort  parameters  of  the  function  boxes  which  would  be  affected  by  the  ECP, 
and  that  may  still  be  reached.  The  ECP  processor  itself  will  include  function 
boxes  to  emulate  any  rework  needed  as  a  result  of  each  ECP.  The  time  and 
effort  impacts  will  have  to  be  treated  as  widely  varying  random  variables 
which  reflect  empirical  data  gleaned  from  other  like  contracts. 

8.1.4  Output  Reports 

The  Simulator  is  planned  as  a  general  tool  for  the  support  of  acquisition 
programs  that  include  software  development.  As  such  it  can  be  useful  for 
project  planning,  proposal  evaluation,  contract  monitoring,  acquisition 
strategy  research,  and  development  strategy  research.  The  users'  need  for 
information  will  reflect  the  specific  application  intended.  While  the  initial 
Simulator  version  will  produce  limited  reports,  future  versions  will  provide 
many  more.  For  this  reason,  the  Simulator  will  be  designed  to  provide 
different  types  of  output  reports,  and  will  allow  operator  inputs  to  specify 
and  delimit  specially  desired  data. 


8.2  PLANNED  FY  80  ACCOMPLISHMENTS 

The  completion  of  the  Simulation  Model  remains  a  multi-year  effort  during 
which  a  sequence  of  increasingly  capable  Models  will  be  developed  and  put  into 
use.  Assuming  funding  at  the  level  of  2  MTS  and  appropriate  computer  time,  we 
plan  to  do  the  following  in  FY  80.  The  project's  overall  plan,  covering  past, 
current,  and  future  work,  is  discussed  in  Section  8.3. 

The  principal  Simulator  developments  planned  for  FY  80  are  completion  of 
a  Demonstration  Model  (MO)  and  establishment  of  a  firm  foundation  for  the 
Prototype  Model  (Ml).  In  addition,  the  Model  Definition  work  will  be  extended 
in  depth. 

a.  Demonstration  Model  (MO) 

(1)  Simulator  Model  0  will  be  completely  coded,  compiled,  integrated 
and  checked  out. 

(2)  Model  quantification  will  be  completed;  parameter  value 
estimates  will  be  provided  for  all  LoSim  function  boxes  listed 
in  Appendix  B  tables.  Parameter  calibration  will  begin  after 
Simulator  Model  0  becomes  available,  but  only  as  time  permits. 

(3)  Model  0  will  be  demonstrated,  but  no  formal  test  (e.g.,  FQT)  is 
planned. 

(4)  Model  definition  will  be  completed  by  general  refinement  of  the 
LoSim  logic  (Figure  A-2).  Also,  the  Process  Flow  Expansion 
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diagram  (Figure  A-3)  and  the  Amplification  Notes  (Table  A-2), 
will  be  completed. 

b.  Prototype  Model  (Ml) 

(1)  The  specific  capabilities  to  be  included  within  Model  1  will  be 
selected  and  functionally  specified. 

(2)  Process  Flow  Diagrams  will  be  updated  as  necessary  to  reflect 
the  added  capabilities . 

(3)  Estimated  parameter  values  will  be  obtained  for  any  functions 
modified  or  added. 

(4)  The  Simulator  design  will  be  updated  to  include  the  new 
capabilities . 

(5)  The  new  capabilities  will  be  implemented  (coded,  compiled, 
integrated,  etc.)  to  the  extent  that  available  resources  permit. 

8.3  OVERALL  PLAN 

A  modest  amount  of  effort  has  been  devoted  to  outlining  an  overall 
strawman  plan  for  the  development,  pilot  application,  and  installation  of  the 
Process  Model,  including  work  in  future  years.  This  outline  is  presented  in 
Table  1,  Outline  of  Strawman  Software  Acquisition  Process  Model  Development 
Plan.  It  needs  more  work  to  become  a  full-fledged  plan,  and  essential 
concurrence  by  all  concerned  organizations  to  become  a  viable  one. 
Nevertheless,  we  hope  that  the  outline  will  help  avert  misunderstandings  about 
our  goals,  our  approach,  and  the  scope  of  our  current  effort.  We  also  intend 
it  to  help  guide  that  effort,  and  believe  that  it  should  lead  to  more 
realistic  expectations  and  financial  planning. 

Our  outline  draws  extensively  on  ideas  contained  in  Sections  2  and  6  of 
the  1  April  1979  draft  AFSC  Software  Cost  Estimation  Working  Group  (SCEWG) 
Research  Management  Plan  (SCEWG79),  prepared  by  Capt.  J.  A.  Duquette, 

ESD/ACCE,  based  on  review  of  an  earlier  version  by  the  ESD/SCEWG.  However,  we 
have  restructured  and  revised  that  plan's  elements,  and  have  rescheduled  the 
revised  results,  in  preparing  our  plan  outline. 

The  strawman  plan  outline  has  several  prime  characteristics,  all  of  which 
we  deem  essential  to  the  Model's  successful  development  and  transition  to 
operational  use.  These  characteristics  are: 

a.  definition,  development,  and  trial  application  of  several  successive, 
modest,  versions  of  the  Model; 

b.  incorporating  experience  with  earlier  versions  into  later  versions; 

c.  strong  emphasis  on  written  application  guidance  and  expert  support  of 
Model  operation  and  maintenance;  and 


d.  formal  review  and  evaluation  of  application  experience  to  assess 
progress,  to  assure  operational  effectiveness,  and  to  help  guide  new  version 
development. 

Table  1  lists  the  major  tasks  and  first-level  subtasks  of  the  strawman 
plan  outline,  including  tasks  already  accomplished  as  well  as  the  FY  80  tasks 
described  in  Section  8.2.  It  also  incorporates  a  rudimentary  schedule  for 
each.  The  plan  outline  has  been  devised  around  four  potential  Software 
Acquisition  Process  Model  versions.  Table  2,  Tentative  Later  Process  Model 
Version  Characteristics,  presents  a  number  of  characteristics  of  the  last 
three  versions,  as  we  currently  envision  them.  In  addition,  each  version  will 
undergo  improvement  during  its  pilot  application,  and  the  third  and  fourth 
versions  will  change  during  their  routine  (production)  application. 

A  principal  objective  of  the  plan  outline  is  useful  application  of  the 
Process  Model  at  the  earliest  feasible  time.  Another  gnal  is  assuring 
adequate  transfer  of  experience  with  early  versions  into  the  designs  of  their 
successors.  A  third  aim  is  to  define  versions  of  modest  difficulty,  each  of 
which  can  be  developed  with  reasonable  effort  and  risk,  and  each  of  which  can 
replace  its  predecessor  smoothly. 

The  versions  identified  in  Table  2  and  the  schedule  in  Table  1  are  based 
on  planning  to  date.  More  thorough  description  of  each  subtask  may  reveal 
additional  opportunities  for  overlapping  tasks,  but  may  also  show  additional 
constraints  on  overlapping.  After  general  agreement  is  reached  on  this 
outline  or  a  revised  one,  a  plan  based  on  it  should  be  developed,  as  follows. 
The  outline  should  be  expanded,  the  subtasks  should  be  described,  and  their 
mutual  information  requirements  should  be  elicited.  The  results  should  be 
checked  for  precedence  conflicts.  The  individual  subtasks ’  resource 
requirements  should  be  estimated,  and  an  improved  plan  (including  a  revised 
schedule)  should  be  prepared,  consistent  with  available  resources.  This  plan 
should  be  completed  and  agreed  on  before  firm  commitments  are  made  about  FY  81 
work,  and  updated  at  roughly  six-month  intervals  thereafter. 

Subsequent  paragraphs  contain  preliminary  information  about  selected 
portions  of  the  plan. 

8.3.1  Feasibility  Analysis 

We  completed  this  task  in  November  1978.  Its  chief  product  is  an 
internal  report  entitled  "ESD  Software  Acquisition  Process  Model  Concept  & 
Feasibility. " 

8.3.2  Demonstration  Model  (MO) 


This  is  the  first  of  the  four  Model  versions.  Its  development  should 
validate  the  Simulator  design  concepts  (as  described  herein),  establish  basic 
capability  to  which  desired  improvements  may  be  added,  and  provide  valuable 
simulation  experience. 


a.  MO  Definition 


This  work  is  completed.  This  report  outlines  Model  0's  structure  and 
capabilities . 

b.  MO  Development 

This  task  is  currently  in  process.  This  report  describes  the  results  to 
date  and  our  plans  for  its  completion. 

8.3.3  Prototype  Model  (Ml) 

a.  Ml  Definition  and  Preliminary  Design 

The  functions  planned  for  inclusion  in  Ml  are  indicated  in  Table  2.  A 
preliminary  design  for  implementing  these  functions  will  lead  to  a  decision 
during  FY  80  as  to  which  functions  are  appropriate  for  Ml. 

b.  Ml  Development 

We  plan  to  start  this  task  in  FY  80.  The  effort  will  consist  of  the 
following  subtasks: 

(1)  Detailed  Design 

We  see  this  as  a  conventional  computer  program  design  subtask. 
However,  we  plan  only  informal  design  reviews,  since  a  computer 
program  of  this  modest  magnitude  needs  no  elaborate  PDR  &  CDR. 

(2)  Coding  &  Checkout 

As  usual,  this  will  cover  coding,  compilation,  and  unit  testing. 

(3)  Informal  Test  &  Integration 

This  subtask  will  cover  integration  and  testing  of  Simulator 
Model  1  as  a  whole.  For  a  program  of  this  modest  size  and 
difficulty,  developed  under  close  customer  observation,  we  plan 
no  formal  testing  (i.e.,  PQT  or  FQT) . 

(4)  Preparation  of  Products 

Selected  output  reports,  related  inputs,  code  listings,  and  a 
set  of  computer  program  operating  instructions  will  be  prepared 
and  delivered.  The  computer  program  source  code  will  also  be 
delivered  on  magnetic  tape.  The  computer  program  operating 
instructions  will  explain  the  mechanics  of  specifying  the 
program's  inputs  and  running  it. 


c.  Ml  Pilot  Application  Preparation 

Preparation  for  the  initial  pilot  application  entails  the  following: 

(1)  helping  ESD  to  locate  existing  ESD  acquisition  programs  which 
might  benefit  from  trial  Prototype  Process  Model  application; 
the  effort  will  include  preparation  and  frequent  delivery  of  a 
briefing  explaining  why  a  PO  should  consider  Prototype  Process 
Model  use; 

(2)  helping  ESD  to  negotiate  a  mutually  satisfactory  arrangement  for 
an  initial  pilot  application  with  one  of  these  acquisition 
program  POs ; 

(3)  developing  a  technical  methodology  for  effective  PO  use  of  the 
Prototype  Process  Model;  and 

(4)  together  with  pilot  PO  personnel,  developing  operating 
instructions  for  use  of  the  Model. 

d.  Ml  Pilot  Application 

This  task  covers  the  application  itself,  and  the  required  support.  Such 
support  includes:  (1)  helping  PO  personnel  to  define  Model  data  and  logic 
changes;  (2)  Simulator  execution,  maintenance,  and  modification  during  pilot 
application;  (3)  interpreting  results  to  PO  personnel;  and  (4)  collecting  data 
for  pilot  application  evaluation. 

8.3.4  Initial  Operational  Model  (M2) 

This  is  the  first  of  the  four  proposed  Process  Model  versions  (see  Table 
2)  for  which  we  propose  routine  operational  use  at  ESD.  We  plan  definition, 
development,  pilot  application  preparation,  and  pilot  application  for  the 
Initial  Operational  Model.  These  subtasks  are  analogous  to  the  corresponding 
Prototype  Process  Model  (Ml)  subtasks.  The  chief  differences  now  apparent 
result  from  the  increased  scope  and  sophistication  of  the  Initial  Operational 
Model,  the  reduced  effort  at  basic  definition  needed,  plus  additional  effort 
that  will  be  required  to  improve  the  accuracy  of  parameter  estimates  and  to 
incorporate  other  improvements  suggested  by  Prototype  Process  Model  pilot 
application  experience. 

8.3.5  Assessment  of  Initial  Pilot  Applications 

We  plan  a  several-month  effort  in  early  FY  82  to  review  the  Prototype 
Process  Model  pilot  application  experience,  and  early  experience  with  the 
Initial  Operational  Model,  to  assure  that  continuation  of  the  effort  into 
routine  operation  is  worthwhile.  Assuming  a  positive  outcome,  the  assessment 
should  also  yield  recommendations  that  influence  such  routine  operation. 


8.3.6  Routine  ESP  Model  Operation  (M2  &  M3) 


To  support  routine  use  at  ESD  of  the  Initial  Operational  Model  (M2)  and 
later  the  Extended  Model  (M3)  we  plan  the  following  tasks. 

a.  Planning,  Training  &  Support 

A  permanent  organization  or  organizations,  responsible  for  coordinating 
SARE  data  collection,  Process  Model  application,  and  analysis  of  related 
information  must  be  established.  We  expect  to  devote  modest  effort  over  an 
extensive  time  period  to  plan  this  organization's  establishment,  and  to 
support  it  technically. 

b.  Model  Operation  at  ESD 

Support  of  PO  applications  is  the  work  envisioned  under  this  subtask. 

c.  Model  Maintenance  &  Modification 

Trouble-shooting  of  the  Simulator  and  its  documentation,  plus  development 
and  installation  of  more  extensive  future  modifications,  are  the  work  planned. 

d.  Transition  to  Other  Users 

Assuming  successful  use  of  the  Model  at  ESD,  its  use  at  other  AFSC 
Product  Divisions  is  expected.  The  effort  planned  for  this  subtask  is  support 
for  such  transition.  This  support  will  include  installation  at  other  sites 
(including  site  adaptation),  and  help  with  initial  applications  at  these 
sites. 

8.3.7  Extended  Model  (M3) 


This  is  the  fourth  and  last  Model  version  included  in  the  plan  (see 
Table  2).  The  effort  foreseen  to  develop  and  install  M3  is  analogous  to  the 
Initial  Operational  Model  subtasks.  The  principal  difference  now  foreseen  is 
additional  work  needed  to  replace  the  Initial  Operational  Model  in  routine 
use,  which  will  probably  mean  an  extended  period  of  parallel  operation. 

8.3.8  Model  Review  and  Evaluation 

This  is  planned  as  a  formal  evaluation  of  the  entire  Model  development 
and  application  effort  by  an  outside  group.  Our  planned  effort  comprises 
support  of  the  evaluation  group. 


Outline  of  Strawman  Software  Acquisition  Process  Model  Development  Plan 


Tentative  Later  Process  Model  Version  Characteristics 
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Run-tine  A  few  standard  outputs  More  outputs,  most  para-  Extensive  output  &  run 

options  only.  meter  values,  no  logic.  controls  to  meet 

selectable  the  defined  needs 

of  the  user. 


SECTION  9 


CONCLUSIONS  AND 
RECOMMENDATIONS 


Effective  system  planning  requires  a  reasonably  accurate  means  for 
estimating  the  cost  and  schedule  for  embedded  software.  This  need  became 
manifest  about  twenty  years  ago  when  the  first  Air  Force  "L"  systems  were 
being  acquired.  Since  then,  system  technology  has  greatly  improved,  enabling 
the  feasibility  of  ever  more  complex  systems.  Software  estimating  and 
management,  however,  remain  rough  empirical  procedures  with  outputs  that  lead 
more  to  surprises  than  to  sound  planning. 

While  the  various  analytic  estimating  techniques  currently  being  used  do 
yield  estimates,  they  have  not  been  able  to  forecast  or  account  for  the  wide 
deviations  in  results  experienced  on  different  but  "similar"  systems.  Because 
of  this  situation,  work  was  begun  on  a  software  estimating  technique  that  is 
based  on  a  simulation  of  the  acquisition/development  process  rather  than  just 
on  the  product  being  acquired.  Though  more  complex  than  the  analytical 
methods,  this  approach  appeared  to  offer  the  prospect  of  better  results. 

Now,  21  months  later,  the  project  is  closer  to  realization  and  the 
prospect  remains  bright.  The  software  acquisition  process  has  been  modeled,  a 
Simulator  design  has  been  formulated  and  partially  implemented,  and  a  plan  has 
been  drawn  up  for  bringing  the  Simulator  into  operational  use.  Based  on  the 
problems  encountered  and  surmounted,  and  the  results  achieved,  the  Simulator 
concept  has  grown  in  feasibility  and  still  offers  the  prospect  of  improved 
software  cost  and  schedule  forecasting. 

In  addition,  during  work  on  the  Software  Acquisition  Process  Model,  it 
became  evident  that  the  device  had  inherent  capability  as  a  general  purpose 
acquisition  management  tool.  Beyond  cost  and  schedule  estimating,  it  could 
aid  in  project  planning,  contractor  proposal  evaluation,  contractor 
monitoring,  and  personnel  training,  as  well  as  in  research  into  the  process  of 
acquiring  and  developing  embedded  software. 

The  results  of  inaccurate  software  estimates  and  dimly  illuminated 
management  decisions  are  manifest  in  the  high  cost  of  acquiring  embedded 
software.  Considering  the  magnitude  of  annual  Air  Force  expenditures  on  such 
software,  improvement  in  the  software  acquisition  process  provides 
considerable  potential  for  cost  savings.  The  Process  Model  described  in  this 
report  offers  such  an  opportunity.  Since  the  project  cost  is  small,  its  cost 
saving  potential  large,  and  the  project  risk  modest,  continuation  of  this  work 
is  prudent  and  recommended. 


APPENDIX  A 


PROCESS  FLOW  DIAGRAMS 
AND  AMPLIFICATION  NOTES 


This  appendix  incorporates  and  explains  the  detailed  diagrams  of  the 
software-related  activities  and  decisions  typical  during  the  Full-Scale 
Development  Phase  of  the  Major  System  Acquisition  Life  Cycle  defined  in 
AFR  800-2.  As  such,  it  presents  the  results  to  date  of  the  Process  Definition 
work,  introduced  in  Section  3  and  discussed  further  in  Section  4. 

First,  Figure  A-l,  Flow  Diagram  Notation,  explains  the  flow  diagram 
conventions.  Table  A-l,  Index  to  Figure  A-2  Connectors  and  Box  Numbers,  is 
provided  to  help  locate  specific  information  in  the  multi-page  LoSim  flow 
diagrams.  Figure  A-2,  Software  Acquisition  Process  Model  LoSim  Activity  Flow, 
depicts  in  200+  connected  activities  and  decisions,  the  software-related 
functions  of  the  entire  Full-Scale  Development  Phase.  Figure  A-3,  Process 
Flow  Expansion,  shows  selected  portions  of  this  process  in  detail  suitable  for 
training  or  the  development  of  parameter  estimates.  Table  A-2,  Process  Flow 
Diagram  Amplification  Notes,  provides  comments  on  selected  activities  and 
decisions  depicted  in  Figures  A-2  and  A-3.  Finally,  the  abbreviations  used  in 
this  appendix  are  listed  and  defined. 


The  Process  Flow  Diagrams  contain  three  basic  types  of  element;  Function 
Boxes,  Auxiliary  Elements  and  Lines  of  Flow,  as  follows: 


1.  FUNCTION  BOXES 
1 . 1  Shapes 

In  HiSim  and  higher-level  Process  Flow  Diagrams, 
rectangles  are  used  to  represent  all  actions; 
these  diagrams  include  none  of  the  other 
Function  Box  shapes,  shown  below.  In  LoSim  and 
Expansion  Process  Flow  Diagrams,  however, 

- rectangles  (i.e.,  Rectangular  Activity  Boxes) 

are  used  only  to  represent  mainstream  activities 
(i.e.,  activities  of  principal  importance). 


Trapezoidal  Activity  Boxes  are  used  to  represent 
support  activities  in  LoSim  and  Process  Flow 
Expansion  Diagrams.  Both  mainstream  and  support 
activities  consume  time  and  resources. 


In  LoSim  and  Process  Flow  Expansion  Diagrams,  a 
hexagon  depicts  a  Special  Event  Box.  A  Special 
Event  provides  for  special  actions  such  as 
changing  a  frequency,  duration,  or  condition,  at 
a  designated  point  in  the  process  logic. 


In  LoSim  and  Process  Flow  Expansion  Diagrams,  a 
rhomboid  depicts  each  Decision  Box.  A  Decision 
Box  is  any  procedure  which  selects  between  two 
mutually  exclusive  exits.  By  convention,  these 
include  no  time  or  resource  expenditures,  which 
are  included  instead  in  preceding  activities. 


1.2  Labels 


Each  Function  Box  has  a  label,  printed  just  above  the  box.  In  the  HiSim 
diagram  each  label  is  a  one-  or  two-digit  number.  In  the  LoSim  diagram,  each 
HiSim  box  may  be  represented  as  a  network  of  several  boxes;  thus,  each  LoSim 
box  label  is  normally  the  corresponding  HiSim  box  label  suffixed  by  a  letter. 
Similarly,  each  Process  Flow  Expansion  box  label  is  normally  the  corresponding 
LoSim  box  label,  suffixed  by  a  distinguishing  integer. 


Figure  A-l.  Flow  Diagram  Notation 


1.3  Features 


Each  box  may  contain  several  field  designators,  identified  by  corner 
positions  within  the  box  as  shown  by  letters  X,  Y,  Z  and  C,  as  follows: 

X  -  indicates  Doer;  i.e.,  the  organization  responsible  for  the  function: 
A  =  government  (e.g.,  Air  Force),  C  =  Contractor,  B  =  Both. 

Y  -  indicates  Integration  Group:  D  =  Developmental  Integration  Group 
(DIG) ,  T  =  Test  Integration  Group  (TIG) ,  Blank  =  the  function  is  not 
divided  into  Groups . 

Z  -  indicates  the  level  at  which  the  work  is  conducted:  1  =  System,  2  = 
Segment,  3  =  CPCI,  A  =  CPC  (Computer  Program  Component),  5  =  lower 
level  module. 

C  -  is  present  on  any  Decision  Box  used  as  a  counter. 


2.  AUXILIARY  ELEMENTS 


Used  to  indicate  a  specific  point  in  the  process 
flow.  May  be  used  to  show  connection  between 
physically  separated  elements  on  flow  diagrams. 
(A  given  label  must  apply  uniquely  to  only  one 
input  point  in  the  process  flow) . 


2.1  Shapes 

Connector 


Terminus 

CED 


Used  to  mark  a  start  or  end  point  of  a  process 
When  labelled  "fin"  it  marks  the  end  of  the 
specific  flow  path. 


Flag 


Do  for 
each  DIG 


Used  to  annotate  flow  diagrams. 


Figure  A-l.  Flow  Diagram  Notation  (Continued) 


3.  LINES  OF  FLOW 


The  lines  of  flow  have  arrows  to  indicate  direction,  plus  three 
alphabetic  designators,  as  follows: 


N/F/S 


♦ 


♦Start  Logic 

A  =  Logical  "AND"  relationships  (the  input  is  necessary 
to  start  the  box) . 

R  =  Logical  "OR"  relationship  (any  one  of  these  will 
start  a  box;  inputs  of  other  types  may  also  be 
necessary,  however). 

S  =  Start  immediately  (this  input  by  itself  will  start 
a  box) . 

-» Progression  Mode  (PM) 

F  =  Normal  forward  progression 
I  =  Iterative  progression 

C  =  Continue  progression  mode  (F  or  I)  of  predecessor. 

♦  Group  Number  Controller 

N  =  No  group  involvement 
D  =  Increment  DIG  number 
T  =  Increment  TIG  number 
G  -  Retain  predecessor's  Group  number. 


Figure  A-l.  Flow  Diagram  Notation  (Concluded) 
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Figure  A-2.  Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 
Sheet  L  -  Organization,  Management,  Support 


Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 
Sheet  2  -  Top  Level  Design  Through  PDR 


Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 
Sheet  3  -  Detail  Design  Thru  CDR 
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Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 
Sheet  4  -  Code/Compile/Debug/Integrate/Checkout 
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Figure  A-2.  Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 
Sheet  5  -  Test  Plan/Procedures/Dry  Runs 
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Software  Acquisition  Process  Flow  Diagram  -  LoSim  Lev«l 
Sheet  6  -  Conduct  Qual  Tests 


Figure  A-2.  Software  Acquisition  Process  Flow  Diagram  -•  LoSim  Level 
Sheet  7  -  Functional  Configuration  Audit  -  FCA 
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Figure  A-2.  Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 
Sheet  8  -  Product  Specs/Users  Manuals 


Figure  A-2.  Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 
Sheet  9  -  Physical  Configuration  Audit  -  PCA 


Figure  A-2.  Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 

Sheet  10  -  Physical  Configuration  Audit  -  PCA  (Concluded) 


Figure  A-2.  Software  Acquisition  Process  Flow  Diagram  -  LoSira  Level 
Sheet  11  -  System  Test 


Software  Acquisition  Process  Flow  Diagram  -  LoSim  Level 
Sheet  12  -  ECP  Subprocess 
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Process  Flow  Expansion 

Sheet  4  -  Boxes  8A  or  12A,  10A,  IOC  &  10E 


Process  Flow  Expansion 

Sheet  5  -  Boxes  44H,  44B,  46L,  46P,  54E,  54D,  54K,  54L 


FROM  46H 


Process  Flow  Expansion 

Sheet  6  -  Boxes  18M,  18N,  46R,  54D,  or  54L 


Table  A-2 


PROCESS  FLOW  DIAGRAM  AMPLIFICATION  NOTES 


Note  Fig.  A-2  Fig.  A-3 
No.  Sht.  Box  Sbt.  Box 


4A  1  4A3 


1  4C  1  4C3 


lification  Notes 


The  assignment  of  key  personnel  at  the 
initiation  of  a  project  is  generally  a  slow 
process .  Each  person  selected  for  a  new 
project  usually  has  an  existing  assignment 
which  must  be  transitioned  to  a  successor;  the 
successor  may  also  need  to  transition  his  job 
to  another,  etc.  Advance  planning  by  the 
contractor  helps  in  the  personnel  selection 
process  but  the  uncertainties  associated  with 
the  award  and  timing  on  this  contract  (as  well 
as  on  other  contracts  bid)  make  startup  a 
traumatic  event  that  gets  under  way  slowly. 

Just  the  concept  and  general  approach  to  the 
Developmental  Integration  Group  (DIG)  (see 
Section  4.1.5)  plan  are  established  here  to 
provide  a  basis  for  the  status  monitoring  and 
management  plans.  The  grouping  of  specific 
CPCs  into  DIGs  is  established  in  Box  6F.  (See 
Note  11). 

Note  that  both  4A3  and  4S  support  activity  4C3 
(CPDP  preparation)  even  though  the  direct 
connection  isn't  shown  on  the  LoSim  diagram. 
This  feedback  is  shown  indirectly  via  the  4S  to 
4A  to  4C  connection;  this  arrangement  will 
satisfy  the  precedence  needs  of  the  Simulator. 

The  System  Engineering  Management  Plan  (SEMP) , 
the  Test  end  Evaluation  Management  Plan  (TEMP), 
and  the  Computer  Resources  Integrated  Support 
Plan  (CRISP)  are  normally  prepared  during  the 
system's  Validation  Phase.  These  plans  usually 
need  updating  in  the  light  of  the  current 
contract  and  contractor.  This  box  covers  only 
those  portions  concerned  with  software. 

The  Computer  Program  Development  Plan  (CPDP) 
is  generally  addressed  in  the  contractor's 
proposal.  This  activity  covers  the  rewrite  and 
extension  necessary  before  this  plan  can  be  put 
into  effect  contractually. 
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Table  A-2  (Continued) 


Note  Fig.  A-2  Fig.  A-3 

No.  Sht .  Box  Sht .  Box  Amplification  Notes 


5  1  4S  1  4S  Per  Section  4.1.4  this  activity  provides  for  the 

most  general  case  where  the  program  and  test 
support  facilities  are  not  identical,  even 
though  some  portions  of  the  hardware  may  be 
shared  for  both  uses. 


6 


7 


8 


9 


1  60A  2  60A1-  A  need  to  build  support  software  will  significantly 

60A7  increase  this  activity's  elapsed  time  over  that 
otherwise  required.  Parameter  values  for  this 
activity  must  be  selected  to  reflect  the  actual 
(or  expected)  contractual  situation. 


1 


62A  2 


62A11  Any  non-trivial  special  (i.e.,  not 
62A12  specified)  equipment  or  software  which  is  to 
62A13  be  used  to  support  Qualification  Testing  must 
62A17  be  evaluated  to  assure  that  it  is  valid  for  its 
62A18  intended  use.  As  examples,  a  facility  may  be 
62A19  needed  to  emulate  a  non-available  interfacing 
component  (hardware  or  software)  or  to  produce 
radar  returns  representing  a  flying  aircraft, 
etc.  Any  deliverable  test  support  component 
would  not  be  processed  in  these  boxes  because 
its  validity  would  be  established  in  the  tests 
associated  with  its  acceptance  by  the 
government. 


1  4G 
4J 
4L 
4M 


The  management  plans  are  frequently  resolved  at 
the  first  full-scale  overall  Program  Management 
Review  (PMR) ,  as  shown.  Instead,  they  can  be 
treated  at  a  separate  meeting  (if  they  become 
urgent  issues),  or  without  a  meeting  (by  mail 
and  phone)  if  not  controversial;  the  process 
parameters  can  be  adjusted  to  cover  any 
expected  case. 


1  66B 

66D 
66F 
9  80D 


PMRs  are  generally  conducted  on  a  periodic 
basis  (e.g.,  monthly  or  bimonthly)  throughout 
the  entire  contractual  period.  They  are  shown 
here  because  the  preparation  and  conduct 
activities  consume  considerable  manpower  on  an 
intermittent  basis  and  thereby  can  impact  the 
development  process.  Note  that  a  Special  Event 
Box  (80D)  will  cause  the  PMR  activity  to  stop 
at  the  start  of  PCA. 
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Table  A-2  (Continued) 


(Note  Fig.  A-2  Fig.  A-3 

No-  Sht.  Box  Sht .  Box  Amplification  Notes 


)0 


11 


12 


2  6A  3  All 

6JD 


2  61 


2  6G 
6H 
61 


The  design  and  evaluation  activities  shown  are 
representative  of  those  conducted  on  many  projects; 
they  are  not  intended  as  an  all  inclusive  set. 

In  general,  the  overall  design  is  sampled  at  a 
moderate  depth  while  design  areas  that  are 
perceived  to  be  risky,  difficult,  or  innovative 
are  given  emphasis. 

Here  the  specific  Developmental  Integration  Groups 
(see  Section  4.1.5)  comprising  the  CPCI  are 
defined . 

The  design  activities  conducted  prior  to  these 

boxes  are  global  in  that  they  include  the  overall  CPCI 

at  a  fairly  gross  level.  They  establish  that  the 

overall  system  concept  is  feasible  and  can 

accord  with  space,  timing,  and  other 

restrictions.  In  these  boxes,  the  capabilities 

to  be  provided  in  each  DIG  (see  Section  4.1.5) 

are  designed  to  a  depth  necessary  to  show  that 

the  approach  for  each  specific  function  is 

feasible . 


13  2  61  4  8A1 

8A9 


14  2  6M 

8C 
12A 
12J 


15  2  8E 

10F 
12E 
12H 


Once  the  contractor  establishes  the  adequacy 
of  his  design  (in  box  61)  he  must  document  it 
(using  Product  Specification  format)  and  submit 
it  for  government  review  and  approval.  This  is 
shown  via  connectors  LE  and  LC  to  Boxes  20A  to 
20E  on  Sheet  8  of  Figure  A-2. 

Even  when  the  PDR  results  (in  Box  8C)  are  satisfactory, 
there  are  generally  a  number  of  specific  deficiency 
areas  noted  during  the  extensive  review.  These  are 
documented  by  the  contractor  in  the  design  review 
minutes  as  items  which  he  agrees  to  correct; 

Box  6M  makes  provision  for  these  corrections. 

The  references  to  Boxes  12A  and  12J  indicate 
that  this  note  also  applies  to  the  CDR  results. 

Each  of  the  Decision  Boxes  branches  on  a  count  rather 
than  on  the  basis  of  probability.  If  the  design  at 
these  points  has  not  been  completed  for  all  the  DIGs, 
this  causes  the  design  process  to  repeat,  but  on 


1G4 


i 


Note 

No. 


I 


Table  A-2  (Concluded) 


Fig.  A-2  Fig.  A-3 

Sht .  Box  Sht.  Box  Amplification  Notes 


18H 

18P 

42B 

42L 

44K 

46N 

48D 


the  next  DIG;  e.g. ,  at  Box  6G.  Note  that  regard¬ 
less  of  the  counter,  further  design  on  the  current 
DIG  continues;  e.g.,  by  transfer  to  connector  D. 
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Appendix  A  Abbreviations 


AF 

Air  Force 

ADEQ 

Adequate 

CCB 

Configuration  Control  Board 

CCI&C 

Code,  Compile,  Integrate  &  Check 

CDR 

Critical  Design  Review 

CDRL 

Contract  Data  Requirements  List 

Cl 

Configuration  Item 

CPC 

Computer  Program  Component 

CPC  I 

Computer  Program  Configuration  Item 

CPDP 

Computer  Program  Development  Plan 

CPT&E 

Computer  Program  Test  &  Evaluation 

CRISP 

Computer  Resources  Integrated  Support  Plan 

CRIT 

Critical 

CTL 

Control 

DEMO 

Demonstrate 

DESCR 

Description 

DEV 

Develop 

DIG 

Developmental  Integration  Group 

DISCREP 

Discrepancies 

DIST 

Distribute 

DOC 

Document 

DSGN 

Design 

ECP 

Engineering  Change  Proposal 

EVAL 


Evaluate 


Appendix  A  Abbreviations  (Continued) 


FACIL 

Facility 

FCA 

Functional  Configuration  Audit 

FIN 

End  of  this  process  flow  diagram  path 

FQT 

Formal  Qualification  Testing 

FUNC 

Functional 

HIERARCH 

Hierarchial 

HWARE 

Hardware 

I&C 

Integration  and  Checkout 

IMPL 

Implementation 

INFO 

Information 

INTEG 

Integration 

LVX 

Level 

MAINT 

Maintain 

MGMT 

Management 

MGR 

Manager 

MISC 

Miscellaneous 

ORG 

Organization 

PCA 

Physical  Configuration  Audit 

PCKG 

Packaging 

PDR 

Preliminary  Design  Review 

PRGM 

Program 

PMR 

Program  Management  Review 

PREP 

Prepare 
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PROB 

PROC 

PROD 

PROG 

PROJ 

REQT 

REVAL 

REVW 

SCHED 

SEMP 

3 'WARE 

SPEC 

SPRT 

STD 

SYS 

SZ 

TECH 

TEMP 

TIG 


Appendix  A  Abbreviations  (Concluded) 


Problem 

Procedure 

Product 

Programming 

Project 

Requirement 

Reevaluation 

Review 

Schedule 

System  Engineering  Management  Plan 

Software 

Specification 

Support 

Standard 

System 

Size 

Technical 

Test  and  Evaluation  Master  Plan 
Test  Integration  Group 

l 
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APPENDIX  B 


MODEL  DEFINITION  DATA 


This  appendix  incorporates  and  describes  the  tables  that  define  the 
software  acquisition  process  logic  and  parameter  values  to  the  Simulator. 

These  tables  are  input  via  a  terminal  to  computer  files  and  may  easily  be 
altered.  As  explained  in  Section  6.4,  the  Simulator  reads  these  files, 
reformats  the  tables,  and  interprets  the  revisions  to  develop  simulation 
results.  Within  broadly-defined  limits  the  tables  may  be  modified,  to 
represent  more  or  less  detail,  differences  in  process  logic,  or  revised 
parameter  values.  Without  needing  revision  itself,  the  Simulator  will 
interpret  the  modified  tables  and  develop  corresponding  simulation  results. 

Table  B-l,  Software  Acquisition  Process  Model  Network  Linkage,  is  a 
tabular  representation  of  the  entire  LoSim  Process  Flow  Diagram  (Figure  A-2). 
Table  B-2,  Software  Acquisition  Process  Model  Activity  Box  Parameter  Data, 
contains  the  manning  and  duration  parameter  value  estimates  thus  far  developed 
for  the  activities  depicted  in  the  LoSim  Process  Flow  Diagram.  Table  B-3, 
Software  Acquisition  Process  Model  Decision  Box  Parameter  Data,  contains 
estimates  of  the  decision  outcome  probabilities  for  all  LoSim  Flow  Diagram 
Decision  Boxes  except  counters.  Table  B-4,  Software  Acquisition  Process  Model 
Counter  &  Special  Event  Box  Parameter  Data,  contains  the  LoSim  Flow  Diagram 
counter  Decision  Box  limits  and  Special  Event  Box  parameters  so  far  defined. 
The  latter  are  mainly  Milestone  identifiers. 

The  columns  of  these  tables,  and  the  values  that  the  data  in  each  column 
may  legitimately  contain,  are  explained  below. 


1.  TABLE  B-l 

Table  B-l  represents  the  Process  Model  network.  It  must  contain  an  entry 
for  each  box  in  the  Process  Flow  Diagram  that  it  represents.  There  is 
currently  an  entry  in  Table  B-l  for  every  box  in  the  LoSim  Process  Flow 
Diagram  (Figure  A-2). 

1 . 1  Box  Data 


a.  Box  ID:  This  is  the  box’s  label  (see  Figure  A-l). 

b.  Box  Type: 

A  =  a  mainstream  Activity  Box. 

B  =  a  branching  box  (i.e.,  a  normal  Decision  Box). 

C  =  a  counter  Decision  Box.  This  is  similar  to  a  type  B  box,  except 
that  the  exit  is  determined  by  whether  an  incrementing  counter 
has  reached  it's  limit;  see  Table  A-2,  Note  15. 


E  =  a  Special  Event  Box.  This  provides  for  special  actions  at 
designated  locations  in  the  process  flow.  These  include 
displaying  Milestones,  resetting  counters,  and  changing  parameter 
values,  and  provide  for  as  yet  undefined  future  needs. 

H  =  a  helping  box  (i.e.,  a  support  Activity  Box).  See  Figure  A-l. 

c.  Box  Lns:  This  is  the  number  of  Table  B-l  lines  used  to  define  this 
box.  This  field  is  included  as  a  programming  convenience.  Multiple  lines  are 
needed  to  accommodate  multiple  predecessors  or  successors  within  the  table 
format. 

1.2  General  Data 

a.  Doer:  This  defines  the  agency  or  agencies  assigned  to  perform  the 
activity  or  to  make  the  decision: 

A  =  Government  (e.g.,  Air  Force) 

C  =  Contractor 

B  =  Both. 

b.  Lvl:  This  defines  the  system  component  level  at  which  the  activity 
is  performed  or  the  decision  is  made,  as  listed  below.  Note  that  a  box  at  any 
level  may  also  be  divided  (or  clustered)  to  conform  with  Integration  Group 
assignments,  described  subsequently. 

1  =  System  level 

2  =  Segment  level  (see  Section  4.1.2) 

3  =  CPCI  level 

4  =  CPC  level 

5  =  Computer  Program  Routi.e  level,  or  lower. 

c.  Box  Grp:  This  defines  the  box's  membership  (if  any)  within  an 
Integration  Group  (see  Section  4. *.5). 

D  =  Developmental  Integration  Group  (DIG) 

T  =  Test  Integration  Group  (TIG) 

N  =  No  Integration  Group. 

d.  Subnet:  A  user  may  assign  the  box  to  any  one  of  up  to  15  Subnetworks 
by  entering  a  number  in  the  range  1-15  in  this  column.  The  Simulator  will 
develop  aggregate  timing  and  cost  data  for  each  Subnetwork  as  well  as  the 
entire  network. 


1.3  Predecessors 


a.  Quan:  This  specifies  the  quantity  of  the  boxes'  immediate 
predecessors.  The  data  required  for  the  first  or  only  predecessor  are 
specified  in  the  "Predecessors"  columns  on  this  line.  If  a  box  has  more  than 
one  predecessor,  the  data  for  its  second  and  any  subsequent  predecessors  are 
stored  in  corresponding  columns  of  successive  lines.  The  "Quan"  column  of 
these  successive  lines  is  blank. 

b.  Box:  This  is  the  Box  ID  (see  paragraph  1.1a)  of  the  predecessor. 

c.  Exit:  This  is  the  predecessor  box's  exit  used  to  reach  this  box: 

Y  =  "Yes"  exit 

N  =  "No"  exit 

S  =  Single  exit. 

d.  Grp:  The  box's  Group  Control  parameter,  used  to  maintain  Group 
(i.e.,  DIG  or  TIG)  number  continuity  and  incrementation  during  network  flow. 

N  =  No  group  involvement 

D  =  Increment  DIG  Number 

T  =  Increment  TIG  number 

G  =  Retain  predecessor's  group  number. 

e.  PM:  A  parameter  used  to  indicate  Progression  Mode  during  box-to-box 
progression.  See  Figure  A-l. 

F  =  Normal  forward  progression 

I  =  Iterative  progression 

C  =  Continue  Progression  Mode  of  predecessor. 

f.  Strt:  Defines  the  combination  of  predecessors  that  must  finish 
before  this  box  may  start. 

A  =  "AND"  relationship.  This  predecessor's  completion  is  a  necessary 
but  not  a  sufficient  condition  for  starting  the  box. 

R  =  "OR"  relationship.  Completion  of  only  one  type  R  predecessor  is 
necessary  to  start  the  box.  Predecessors  of  other  types,  if 
specified,  are  also  required. 

S  =  Start  immediately.  This  predecessor's  completion  by  itself  is 
sufficient  to  start  this  box. 


Note:  If  a  box  has  no  predecessor  (e.g.,  Box  2A)  all  of  its  entry's 
predecessor  fields  are  zero  filled. 

1.4  Successors 


a.  Qnys:  This  specifies  the  quantity  of  the  box's  “Yes"  exit  (or  single 
exit)  immediate  successors. 

b.  Qnn:  This  specifies  the  quantity  of  the  box's  “No"  exit  immediate 
successors . 

c.  Ysbx:  "Yes"  successor  Box  ID.  If  a  box  has  more  than  one  "Yes" 

successor,  their  box  IDs  are  stated  in  successive  lines  in  this  column. 

d.  Nbx:  "No"  successor  Box  ID.  If  a  box  has  more  than  one  "No" 

successor,  their  box  IDs  are  stated  in  successive  lines  in  this  column.  Note 

that  Special  Event  Boxes  may  use  the  Nbx  column  to  list  the  IDs  of  remote 

boxes  which  are  targeted  for  change. 


2.  TABLE  B-2 

Table  B-2  contains  the  parameter  data  for  each  Activity  Box  (box  types  A 
&  H)  in  Table  B-l.  Every  Activity  Box  must  have  a  Table  B-2  entry. 

Tables  B-3  &  B-4  contain  the  parameter  data  for  the  other  box  types. 

2.1  Box  Data 

a.  Box  ID:  This  is  the  box's  label,  which  must  be  identical  to  its 
Table  B-l  Box  ID  (see  paragraph  1.1a). 

b.  Box  Type:  This  must  be  the  same  as  the  Table  B-l  entry's  Box  Type 
(see  paragraph  1.1b). 

c.  Box  Grp:  Identical  to  the  Table  B-l  entry's  Box  Grp  (see  paragraph 
1.2c). 

2.2  Manpower 

Manpower  is  subdivided  into  five  categories  of  work  for  contractor 
personnel,  and  three  for  government  personnel,  as  explained  below.  Note  that 
management  personnel  are  not  assigned  to  specific  activities.  Instead, 
manpower  and  dollar  costs  representing  a  given  management  structure  will  be 
sustained  for  the  project  as  a  whole,  or  for  designated  parts  of  it. 
Management  personnel  effort  is  not  shown  for  specific  boxes  even  if  the  work 
is  largely  done  by  such  persons. 
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The  table  contains  a  pair  of  columns  for  each  manpower  category;  i.e.: 


a.  Contractor 

Sys  =  System  engineers  and  analysts 
Dsgn  =  Designers  (junior  and  senior) 

Prgm  =  Computer  programmers 
Test  =  Software  test  engineers 

Sprt  =  Support  personnel;  e.g.,  writers,  operators, 
maintenance  persons. 

b.  Government 

Dev  =  Developing  Command  (e.g.  ESD) 

Usr  =  Using  Command  (e.g.  TAC) 

Sprt  =  Supporting  Command  (e.g.,  AFLC). 

For  each  manpower  category  the  first  column  contains  the  (integral) 
number  of  personnel  of  that  type  estimated  necessary  to  complete  the  activity. 
The  second  column  allows  assignment  to  be  on  a  part-time  basis;  i.e.,  by 
showing  the  proportion  of  their  time  (in  tenths)  these  personnel  apply  to  this 
task.  Here  blank  =  full  time. 

c.  Iterate  Factor: 

Many  tasks  may  need  to  be  repeated  because  the  results  achieved  on  the 
first  pass  were  not  adequate  to  meet  subsequent  needs  or  review  criteria. 

Since  the  work  required  on  subsequent  passes  usually  involves  fewer  persons, 
these  three  columns  each  contain  a  factor  (from  0  to  10)  representing  the 
number  of  tenths  by  which  the  original  number  of  persons  in  each  of  the 
manpower  columns  (as  specified  for  the  first  pass)  must  be  multiplied  to 
obtain  the  manpower  needed  respectively  on  the  second,  third,  and  fourth  or 
later  iteration  of  the  activity. 

2.3  Durations 


a.  Days:  The  first  duration  column  contains  the  mean  duration  of  the 
activity,  in  work  days. 

b.  It  Fctr:  The  next  three  columns  each  contain  a  factor  (from  0  to  10) 
representing  the  number  of  tenths  of  the  first  iteration's  duration  (i.e., 
days  column)  required  to  complete  the  second,  third,  and  fourth  or  later 
iteration,  respectively. 
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2.4  Wait 


This  field  may  contain  a  waiting  time  (in  days)  before  the  activity  may 
begin.  The  action  may  begin  only  after  the  wait  period  has  completed;  the 
Wait  itself  starts  after  all  predecessor  conditions  are  satisfied.  If  blank, 
no  Wait  is  required. 

2.5  Notes 


This  column  refers  to  the  notes  listed  within  Table  A-2,  Process  Flow 
Diagram  Amplification  Notes. 


3.  TABLE  B-3 

Each  Table  B-3  entry  contains  the  parameter  data  for  a  normal  Decision 
Box  (box  type  B)  with  an  entry  in  Table  B-l.  Every  normal  Decision  Box  in 
Table  B-l  must  be  represented  by  a  Table  B-4  entry. 

3. 1  Box  Data 

These  fields'  definitions  are  given  in  paragraphs  2.1a-c. 

3.2  Yes  Exit  Probability 

These  four  columns  contain  the  probabilities  (each  multiplied  by  100)  of 
taking  the  "Yes"  exit  on  the  first  four  iterative  passes  through  the  Decision 
Box;  see  paragraph  2.2c.  The  leftmost  column  provides  first  pass  probability. 
The  rightmost  column  probability  will  be  used  repeatedly,  if  the  box  is 
iterated  more  often  than  four  times. 

3.3  Wait 


See  paragraph  2.4. 

3.4  Notes 


See  paragraph  2.5. 


4.  TABLE  B-4 

This  table  contains  an  entry  for  each  counter  Decision  Box  (type  C)  and 
each  Special  Event  Box  defined  in  Table  B-l.  Every  such  box  must  be 
represented  by  a  Table  B-4  entry. 

4.1  Box  Data 


These  fields'  functions  are  given  in  paragraph  2.1a-c. 
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4.2  Funct 

Three  types  of  function  have  thus  far  been  allocated  to  Special  Event 
Boxes : 

M  =  Milestone.  The  contents  of  the  Event  Label  column  (a  Milestone 
name)  will  be  output  on  schedule  reports  for  each  Special  Event 
Box  entered.  Where  "(DTN.NO)"  is  included  in  the  message  the 
Integration  Group  number  at  the  time  of  entry  into  the  box  will 
be  printed  as  part  of  this  output. 

R  =  Reset.  The  stored  data  and  status  of  the  boxes  listed  in  the 

Table  B-l  Nbx  column  will  be  reset  for  any  Integration  Group  (DIG 
or  TIG)  in  effect  as  of  the  time  the  box  was  entered. 

P  =  Parameter  modification.  The  parameters  given  in  the  Parameter 
column  will  replace  the  existing  values  for  the  boxes  listed  in 
the  Table  B-l  Nbx  column. 

4.3  Event  Label 

This  contains  the  characters  to  be  output  as  the  Milestone  name  for  a 
Milestone- type  Special  Event  Box. 

4.4  Parameter 


This  column  identifies  the  parameter  which  is  to  be  changed  by  a  reset 
(type  R)  or  parameter  modification  (type  P)  Special  Event. 

4.5  Notes 

See  paragraph  2.5. 
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Table  B-2  (Continued) 


52U 

52V 


Table  B-<!  (Continued) 


Table  B-2  (Concluded) 


Table  B-3 


Software  Acquisition  Process  Model 
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Table  B-4 


Software  Acquisition  Process  Model 
Counter  &  Special  Event  Box  Parameter  Data 
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APPENDIX  C 


RATIONALE  FOR  PROCESS 
MODEL  DEVELOPMENT 


This  appendix  repeats  attachment  2  of  MITRE  letter  to  ESD/TOIT  D75-173, 
"Proposed  FY  79  Project  5720  Tasks  in  Software  Resource  and  Cost  Prediction", 
17  July  1978,  because  the  letter  is  unavailable  to  most  readers  of  this 
report.  The  attachment,  entitled  "Advantages  of  Software  Acquisition  Process 
Model  Development,"  is  as  follows. 

FY  79  MITRE  work  to  date  strongly  indicates  substantial  potential 
benefits  from  developing  and  applying  in  selected  ESD-managed  acquisition 
programs  a  simulation  model  of  the  software  acquisition  process .  The  type  of 
software  acquisition  process  model  considered  would  represent  explicitly 
(e.g.,  in  flowchart  form)  the  different  Program  Office  (PO)  and  contractor 
activities  that  ESD-managed  software  acquisition  entails,  and  how  these 
different  activities  interact.  It  would  represent  activity  sequences, 
repetitions  of  such  sequences,  alternatives,  concurrency,  and  delays  (e.g., 
due  to  waits  for  essential  inputs).  In  this  respect  the  model  would  somewhat 
resemble  PERT,  but  without  PERT's  restrictions  on  loop  representation,  etc. 

Besides  this  process  logic,  the  model  would  also  accept,  wherever 
appropriate,  a  definite  parameter  value  or  distribution  of  values  for  each 
activity's  resource  requirements  (e.g.,  manpower,  computer  time),  elapsed 
time,  and  dollar  cost.  The  software  acquisition  process  model  would  also 
accept  parameter  values  representing  the  probabilities  of  transition  among  the 
different  activities;  for  example,  the  chances  of  1,  2,  or  more  document 
review  cycles  could  be  represented.  (An  initial  set  of  these  parameter  values 
would  be  included  so  users  would  need  to  change  only  the  values  pertinent  to 
each  use). 

Early  in  an  acquisition  program's  life  cycle  a  flowchart  of  the 
acquisition  program's  planned  software-related  activities  would  be  developed, 
and  coded  in  a  computer  simulation  language.  The  parameter  values  associated 
with  the  program's  activities  and  transitions  among  activities  would  also  be 
estimated  and  put  into  the  model.  These  flowcharts  and  local  estimates  would 
be  prepared  for,  and  reviewed  by,  the  Program  Manager  (PM),  based  on  current 
acquisition  program  plans.  The  model  thus  defined  would  then  undergo  computer 
simulation  to  develop  initial  predictions  of  the  acquisition  program's  overall 
software-related  requirements,  schedules,  and  dollar  costs,  in  total,  by 
phase,  and  by  function.  In  addition,  the  simulation  would  summarize  delays, 
by  function,  and  other  indicators  of  potential  problems.  The  software 
acquisition  process  model  logic  could  be  modified,  the  parameter  values 
altered,  and  the  simulation  program  rerun,  to  explore  the  effects  of 
alternative  policies  and  the  overall  estimates'  sensitivities  to  particular 
local  estimates. 

Such  early  use  of  a  software  acquisition  process  model  would  force  early 
definition  of  concrete  plans,  and  early  development  of  specific  local 
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estimates.  This  alone  would  be  a  major  benefit  of  using  a  software 
acquisition  process  model  in  an  acquisition  program.  While  the  process  model 
concept  mandates  no  specific  level  of  detail,  and  in  fact  allows  that  level  to 
be  varied  at  will,  the  effort  necessary  to  develop,  and  to  review  at  any 
reasonable  level,  acquisition  program  logic,  local  estimates,  and  simulation 
results,  would  be  sure  to  expose  some  vagueness,  ambiguity,  and  oversight, 
which  if  then  corrected  would  prevent  serious  later  problems. 

In  addition,  software  acquisition  process  model  simulation  results  would 
generally  be  more  accurate  than  estimates  derived  by  more  general  methods 
(e.g.,  using  typical  parametric  cost  estimation  models),  because  the  former 
would  reflect  the  specific  acquisition  program's  plans  and  policies.  Where 
desired,  the  simulation  program  could  also  develop  envelopes  of  values  around 
these  results,  as  measures  of  their  uncertainty. 

Perhaps  most  important,  the  simulation  results  would  be  credible ,  because 
they  would  be  based  on  flowcharts  and  local  estimates  understood  and  approved 
by  PO  management.  The  software  acquisition  process  model  logic  could  normally 
be  verified  by  flowchart  inspection.  The  local  estimates  would  in  some  cases 
be  verifiable  immediately  as  relatively  simple  facts,  or  as  assumptions;  in 
other  cases  verification  would  require  investigation,  but  usually  would  be 
relatively  straightforward  because  it  would  involve  a  local  variable.  In 
contrast,  typical  parametric  models  available  today  are  mysterious  because 
they  represent  no  well-defined  mechanisms,  and  unverif iable  (at  desirable 
levels  of  accuracy)  because  appropriate  data  for  their  rigorous  statistical 
testing  is  unavailable.  Only  a  systematic  data  collection  effort  (like  the 
planned  ESD  Software  Data  Reporting  System),  applied  over  many  years,  can 
substantially  mitigate  this  problem. 

As  an  acquisition  program  evolves,  actual  resource  expenditures,  elapsed 
times,  and  dollars  spent  become  known,  and  the  actual  (vs.  planned)  logic  of 
past  activities  becomes  clear.  Based  on  such  experience  a  PO  changes  its 
acquisition  program  plans,  and  updates  its  predictions  of  future  activities' 
costs.  In  an  acquisition  program  application,  the  software  acquisition 
process  model  would  be  revised  at  convenient  intervals  to  reflect  past  event 
logic,  actual  expenditures,  actual  elapsed  times,  revised  plans,  and  new 
estimates  based  on  all  of  the  above.  After  each  such  set  of  changes,  the 
revised  acquisition  process  model  would  provide  the  PO  with  a  series  of 
overall  predictions,  generally  of  increasing  accuracy.  Each  of  these  would  be 
based  on  the  best  experience  then  available,  and  on  then  current  plans. 

Like  the  PO's  original  version  of  the  software  acquisition  process  model, 
any  subsequent  version  could  be  altered  and  simulated  to  explore  the  effects 
of  proposed  policy  changes  and  different  local  estimates.  In  this  way,  the 
model  could  become  a  powerful  aid  to  source  selection,  provided  the 
acquisition  program's  RFP(s)  directed  the  offerors  to  provide  the  necessary 
input  in  their  proposals,  and  that  the  source  selection  schedule  allowed 
enough  time  for  the  necessary  model  modifications,  parameterization,  and 
simulation  runs.  To  help  in  source  selection,  the  PO's  current  version  of  the 
software  acquisition  process  model  would  be  modified  and  simulated  (where 
possible)  to  reflect  each  offeror's  plans  for  software-related  work,  and 
parameterized  to  reflect  his  resource,  elapsed  time,  and  dollar  cost 
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estimates.  These  activities  should  reveal  omissions,  inconsistencies,  and 
unexpected  implications  that  current  proposal  review  methods  rarely  catch. 

The  model  would  also  be  modified,  parameterized,  and  simulated  to  reflect  the 
Government ' s  best  assessment  of  the  activities,  resources,  elapsed  times,  and 
dollar  costs,  necessary  to  do  the  RFP-specified  work,  based  on  the 
Government's  understanding  of  each  offeror's  proposed  approach,  modified  for 
completeness,  consistency,  and  realism.  The  results  of  this  modification  and 
simulation  would  considerably  improve  the  realism  of  contract  negotiations  by 
sensitizing  the  Government  negotiators  to  proposal  flaws  now  often  overlooked. 
For  similar  reasons,  Source  Selection  Board  recommendations  would  be  improved. 

After  source  selection,  the  PO's  then  standard  model  version  would  be 
altered  to  reflect  the  selected  contractor's  development  approach.  It  would 
subsequently  be  updated,  as  indicated  earlier,  to  reflect  actual  contractor 
performance,  and  used  to  predict  the  effects  of  proposed  changes  in  contract 
scope,  contractor  organization,  and  contractor  resource  allocation.  For 
example,  a  contractor's  justification  of  his  estimates  to  implement  a  proposed 
contract  change  (e.g.,  an  ECP)  could  be  subject  to  verification  by  PO  software 
acquisition  process  model  use.  As  a  result,  excessive  ECP  implementation  cost 
estimates,  now  a  common  problem,  would  often  be  avoided. 

Besides  these  specific  acquisition  program  applications,  a  software 
acquisition  process  model  could  greatly  aid  investigation  of  the  quantitative 
effects  on  resource  requirements,  elapsed  times  and  dollar  costs  of  varying 
the  program  development  or  maintenance  environment  (e.g.,  by  the  use  of 
structured  programming,  improved  compilers,  or  other  development  tools),  of 
imposed  schedules,  of  computer  program  type  and  complexity,  and  of  many  other 
factors.  In  this  type  of  use  a  software  acquisition  process  model  would  be 
altered  or  parameterized  to  reflect  each  factor  or  appropriate  combination 
of  factors.  The  model  would  then  be  simulated,  the  effects  noted,  and  the 
causes  investigated.  Multiple  runs  using  different  estimates  could  establish 
the  sensitivity  of  the  overall  process,  or  of  important  subprocesses  (e.g., 
detailed  design,  coding,  test  and  integration),  to  different  assumed  values  of 
the  factor(s)  under  investigation.  In  effect,  this  kind  of  model  application 
would  provide  a  controlled  experimental  environment  not  achievable  in 
practice.  It  could  be  supplemented  by  limited  actual  experiments  to  obtain 
realistic  parameter  values. 

Probably  the  most  important  near  term  use  of  this  kind  would  apply  the 
software  acquisition  process  model  to  help  revise  and  redefine,  in  FY  79  and 
later,  the  prototype  set  of  software-related  cost  reporting  elements  defined 
under  a  FY  78  MITRE  task.  One  basic  objective  of  this  work  is  to  define  a 
minimum  number  of  elements  that  nevertheless  capture  and  distinguish  all 
important  kinds  of  software-related  resource  expenditures,  elapsed  times,  and 
costs.  The  prototype  element  set  definitions  are  being  established  in  large 
part  as  a  result  of  a  few  persons'  software  development  and  software 
acquisition  management  experience.  While  widespread  critical  review  will 
doubtless  suggest  many  improvements,  and  while  planned  pilot  application  will 
identify  others,  these  activities  could  nevertheless  overlook  important 
deficiencies . 


Software  acquisition  process  sradeling  could  significantly  help  the 
element  redefinition  process.  As  flowcharts  of  the  principal  kinds  of 
software-related  activity  are  developed,  and  as  estimates  are  prepared  of  the 
individual  activities  and  their  transition  probabilities,  it  often  becomes 
clear  what  activities  should  be  grouped  together,  and  what  their  approximate 
relative  magnitudes  are.  This  has  already  occurred  in  some  cases  during  our 
preparation  of  sample  software  acquisition  process  flowcharts  during  the  FY  78 
model  feasibility  study.  Definition  of  other  groups  of  processes  is  expected 
to  yield  further  insights  of  this  kind.  Actual  model  simulation  would  be  even 
more  effective.  Since  the  software  cost  reporting  elements  should  seldom  be 
changed  once  the  Software  Data  Reporting  System  is  widely  applied,  these 
elements  should  be  well  defined  and  reasonably  stable  before  then.  In  the 
absence  of  a  respectable  data  base  (which  the  Software  Cost  Reporting  System 
is  designed  to  collect),  the  software  acquisition  process  model  seems  a  very 
promising  aid  to  judgment  and  limited  experience  in  defining  the  elements 
effectively. 
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APPENDIX  D 


PARAMETER  VALUE 
CALIBRATION 


This  Appendix  describes  the  numerical  methods  used  to  calibrate  the 
activity  duration  data,  and  provides  the  detailed  results  obtained. 

Figure  D-l,  Activity  Duration  Calibration,  shows  the  calibration  procedures 
applied  to  each  unit  sequence  of  the  mainstream  developmental  activity  for  the 
contract  period  up  through  Formal  Qualification  Testing  (FQT).  Figure  D-2, 
Activity  Duration  Calibration  Summary,  shows  the  sequential  relationship 
between  the  unit  sequences  and  the  consequent  overall  timing.  Each  of  the 
figures  is  further  described  below. 

As  shown  in  these  figures,  four  different  activity  times  were  used  or 
developed.  These  are: 

a.  Initial  value  -  the  estimated  elapsed  time  initially  established. 

b.  Initial  iterated  value  -  the  initial  estimated  time,  which  has  been 
lengthened  to  include  the  effect  of  iteration. 

c.  Calibrated  value  -  the  time  assigned  as  a  result  of  the  calibration 
procedure. 

d.  Calibrated  iterated  value  -  the  calibrated  time  with  the  lengthening 
effect  of  iteration  added. 

In  Figure  D-l  the  overall  process  was  divided  into  a  contiguous  set  of 
unit  sequences.  Boundaries  between  the  sequences  were  selected  to  avoid  (or 
minimize)  the  interruption  of  any  iteration  loops.  Each  sequence  was  then 
timed  by  following  its  flow  paths  in  conformance  with  the  Decision  Box 
probability  (pY)  data.  This  was  done  by  assuming  a  sample  of  100  runs  during 
which  a  count  was  kept  of  how  many  times  each  Activity  Box  (A-box)  was  entered 
per  each  iteration  count.  This  permitted  the  total  time  spent  in  each  A-box 
to  be  calculated.  This  is  shown  (per  iteration)  on  the  diagram.  By  dividing 
the  total  time  by  the  number  of  runs  (i.e.,  100),  the  "iterated"  time  for  each 
A-box  was  determined.  Thus  the  most  likely  time  to  traverse  each  unit 
sequence  became  the  sum  of  the  iterated  times  for  the  A-boxes  which  form  the 
unit  sequence. 

Figure  D-2  shows  the  overall  flow.  Sheet  1  represents  the  workdays  for 
each  sequence  based  on  calibrated  activity  times  (without  iteration).  The 
number  in  parentheses  represents  the  cumulative  work  days  (based  on  the 
longest  path)  for  each  sequence.  Sheet  2  depicts  the  same  kind  of  information 
using  the  initial  iterated  times.  The  same  information,  but  based  on 
calibrated  iterated  time,  is  shown  in  Figure  10  of  the  report  proper. 


Figure  D-l.  Activity  Duration  Calibration 
Sheet  1 


Figure  D-l.  Activity  Duration  Calibration 
Sheet  2 


Figure  D-l.  Activity  Duration  Calibration 
Sheet  3 


Figure  D-l.  Activity  Duration  Calibration 
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Figure  D-l.  Activity  Duration  Calibration 
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Figure  D-2.  Activity  Duration  Calibration  Summary 

Sheet  1  -  Calibrated  Activity  Durations,  With  No  Iteration 
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Figure  D-2.  Activity  Duration  Calibration  Summary 

Sheet  2  -  Initial  Estimated  Activity  Durations,  With  Iteration 


APPENDIX  E 


SIMULATION  PROGRAM  LISTING  EXTRACTS 


This  appendix  provides  extracts  from  the  compilation  listing  of  the 
simulation  computer  program  thus  far  completed. 

The  computer  printouts  provided  herein  comprise  the  following  three 
portions: 

a.  The  portions  of  the  Global  Data  Base  prepared  to  date  are  defined 
within  the  program  Preamble.  This  appears  on  pages  1-6  of  the  listing;  these 
page  numbers  appear  on  the  top  right  corner  of  each  page. 

b.  The  Data  Input  Processor  routines  prepared  to  date  are  included  on 
listing  pages  11-34.  Brief  descriptions  and  definitions  of  local  variables 
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APPENDIX  F 


EVENT  NOTICE  SPECIFICATION 


1 .  INTRODUCTION 

In  the  Simulation  Conduct  Processor  the  overall  control  of  flow  through 
the  network  will  be  accomplished  by  the  use  of  SIMSCRIPT  II. 5  Event  Notices  to 
control  the  box-to-box  succession  and  to  pass  parameters.  Two  principal  types 
of  Event  Notice  are  required;  each  has  its  own  parameter  list  and  an 
associated  Event  Routine  to  accomplish  its  tasks.  These  Event  Types,  listed 
below,  are  fully  specified  subsequently.  Other  (and  simpler)  Event  Types 
(e.g. ,  to  end  each  simulation  pass,  to  end  simulation)  will  be  specified 
during  FY  80. 

Para  Name  Function 

3.1  Flow  The  Flow  Event  Type  provides  a  standard  mechanism  for 

transitioning  from  a  terminating  box  to  its  successor 
boxes.  A  Flow  Event  is  scheduled  whenever  a  Box.Proc  Event 
Routine  has  completed  its  processing  on  the  designated  box. 
When  invoked,  the  Flow  Event  Routine  performs  termination 
processing  on  the  completed  box  and  then  performs  the 
following  actions  (as  applicable)  for  each  successor  box: 

-  Determination  of  Group  Number  and  Progression  Mode; 

-  Predecessor  Wait  and  Internal  Wait  processing; 

-  Predecessor  Input  Status  updating;  and 

-  Box.Proc  Event  Notice  scheduling. 

3.2  Box.Proc  The  Box.Proc  Event  Type  initiates  a  common  process  for 

simulating  appropriate  actions  whenever  a  process  function 
box  is  ready  to  begin.  The  associated  Event;  Routine 
observes  special  override  instructions,  if  any,  then 
determines  the  box  type,  and  calls  in  the  corresponding  box 
processor  routine.  When  the  box  type  specific  actions  have 
been  completed,  a  Flow  Event  is  scheduled  for  the  box. 


2.  DATA  NOTATION 

Each  of  the  following  Event  Routine  descriptions  makes  extensive 
reference  to  the  Global  Data  Base;  this  is  defined  (as  yet  incompletely)  in 
the  coded  program's  Preamble,  as  listed  in  Appendix  E.  While  the  Event 
Routines  use  the  listing  data  names,  additional  notation  is  provided  below  to 
convey  the  dimensionality  of  much  of  the  data.  This  notation,  which  is 
described  below,  is  intended  as  an  aid  to  the  reader. 


a.  Each  data  name  is  shown  within  parenthesis. 


b.  All  underlined  data  items  are  variables;  all  other  items  remain 
constant  during  the  simulation  run. 

c.  Dimensional  factors  are  indicated  by  small  letters  which  immediately 
follow  the  enclosed  name,  with  commas  being  used  as  separators.  The  following 
factors  are  used: 

g  =  Integration  Group  number  (DTN.No)  dependency 
i  =  iteration  number  (It.Ct)  dependency 

a  =  assigned  manpower  type;  i.e.,  one  of  a  vector  of  manpower 
quantities  needed  for  each  task. 

ra  =  monthly  accumulation  of  data 

p  =  predecessor  list  membership 

s  =  successor  list  membership. 

d.  Several  examples  illustrate  this  usage. 

(CMN)a,g,i 

"CMN"  contains  the  contractor's  manpower  assignment. 

"a"  indicates  that  each  value  belongs  to  a  vector  of  manning  levels 
for  the  different  assigned  types  of  personnel. 

"g"  indicates  that  the  duration  may  be  diffeient  for  each  Integration 
Group  (DTN.No). 

"i"  indicates  that  the  manpower  level  can  change  per  iteration. 
(IDUR)g, i 

"IDUR"  contains  the  duration  assigned  to  an  activity. 

"g"  indicates  that  the  duration  may  be  different  for  each  Integration 
Group  (DTN.No). 

"i"  indicates  that  the  duration  is  also  dependent  on  the  iteration 
count  (It.Ct)  for  the  Group. 
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"PMN"  indicates  the  Predecessor  Input  Status  for  a  box. 


"p"  indicates  that  each  PMN  is  part  of  a  vector  which  is  ordered  on 
predecessor  list  position. 

"g"  indicates  that  Predecessor  Input  Status  is  maintained  for  each 
Integration  Group. 

(FNETCM)a ,m 

"FNETCM"  accumulates  the  contractor  manpower  expended  on  the  Full 
network. 

"a"  indicates  that  each  value  belongs  to  a  vector  per  assigned 
personnel  type. 

"m"  indicates  that  the  data  are  aggregated  into  monthly  packets. 


3.  EVENT  NOTICE  &  COMPUTER  ROUTINE  SPECIFICATION 

3. 1  Flow:  Network  Flow  Effector 

3.1.1  Purpose 

The  Flow  Event  Program  performs  all  necessary  box  termination  activities 
for  the  designated  box  and  then  updates  the  Predecessor  Input  Status  for  each 
box  listed  in  the  designated  set  of  successors.  It  also  performs  Internal 
Wait  processing  and  Event  Notice  scheduling  for  each  successor  (if  any)  whose 
predecessor  input  conditions  have  been  met. 

3.1.2  Event  Scheduling 

A  Flow  Event  Notice  is  scheduled  for  a  designated  box  whenever  the 
Box.Proc  Event  Routine  completes  its  processing  of  that  box. 

3.1.3  Actions 

a.  Box  Termination  Processing 

(1)  The  following  incoming  parameters  are  saved  as  local  variables: 

(a)  Box. Point  becomes  Own. Box:  the  pointer  to  the  box  to  be 
terminated. 

(b)  PMN  becomes  PMP:  the  Progression  Mode  of  the  predecessor 
box. 

(c)  DTNN  becomes  PDTN:  The  Integration  Group  (i.e.,  DTN.No)  of 
the  predecessor. 


(d)  XIT  is  used  to  select  the  designated  list  of  successor 
boxes  (YList  or  NList) . 

(2)  Own. Box  Status  (Box.Stat)g  is  set  to  zero  (OFF). 

(3)  For  Activity  Boxes  only,  all  previously  allocated  resources  are 
released.  If  an  override  was  in  effect  for  the  box 

(Flow. Override  NEq  0),  no  resources  were  used;  therefore  no 
resources  need  to  be  released.  Otherwise,  the  following  actions 
are  taken: 

(a)  Each  element  of  the  contractor  manpower  use  vector 
((CMN)a,i)  is  subtracted  from  the  corresponding  network 
total  ((CMNA)a) . 

(b)  Each  element  of  the  government  manpower  use  vector 
((GMN)a,i)  is  subtracted  from  the  corresponding  network 
total  ((GMNA)a) . 

(c)  Both  of  the  above  manpower  use  network  total  vectors  are 
saved  in  the  sequential  activity  history  file 

(Rsr .History) .  This  file,  which  will  contain  a  timed 
accounting  of  all  resource  transactions,  will  support  the 
creation  of  a  resource  utilization  profile.  In  later 
versions  the  impacts  resulting  from  resource  limitations 
will  be  simulated. 

b.  Successor  Box  (S-Box)  Processing.  For  each  successor  box  (S-Box)  in 
the  Own. Box  successor  list  designated  by  input  Event  Notice  parameter  XIT,  the 
following  actions  are  effected;  if  the  successor  list  is  empty.  Flow  returns 
with  no  further  action. 

(1)  The  successor  list  Box. Pointer  is  used  to  gain  access  to  the 
following  S-Box  data:  the  Box. Type;  its  Group  dependency  (DTN); 
and  its  list  of  predecessors  (PList) . 

(2)  The  S-Box  predecessor  list  (PList)  is  searched  to  find  the 

Own. Box  entry  (Plist  Box. Pointer  =  Own. Box).  The  following  data 
are  extracted: 

(a)  "p"  -  Own. Box  position  on  the  PList 

(b)  (SLA)p  -  Start  Logic  Assignment  (see  Appendix  A-l) 

(c)  (Gp.Ctrl)p  -  Group  Number  (DTN. No)  Controller  (see  Appendix 
A-l) 

(d)  (PMA)p  -  Progression  Mode  Assignment  (see  Appendix  A-l) 

(e)  "n"  -  Own. Box  location  on  the  Predecessor  Input  Status 
Indicator  vector. 
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(3)  The  Integration  Group  number  (DTN.No)  for  this  Predecessor  Input 
update  is  then  obtained  as  follows: 

(a)  If  Own. Box  Group  Dependency  (DTN)  is  "none"(0),  DTN.No  is 
set  to  1;  otherwise: 


(b) 

The  value 

of  (Gp.Ctrl)p  determines  the  Integration  Group 

number  (DTN.No)  as  follows: 

Gp.Ctrl 

DTN.No  = 

Comment 

0 

1 

(S-Box  not  group  dependent) 

1 

PDTN+1 

(Increment  DIG  Count) 

2 

PDTN+1 

(Increment  TIG  Count) 

3 

PDTN 

(Continue  Pred  DTN.No) 

The 

Progression 

.  Mode  (PMN) 

is  then  determined  using  the  S-Box 

(PMA)p  and  the 

local  (PMP) ; 

as  follows: 

PMA 

PMN  = 

Comment 

1 

1 

Forward  Progression  mandated 

2 

2 

Iterative  Progression  mandated 

3 

PMP 

Continue  Pred  Progression 

(5)  If  PMN  =  2  (Iterative  Progression)  the  action  continues  at 
paragraph  3.1.3b(ll),  Iteration  Count  Actions. 

(6)  If  PMN  =  1  (Forward  Progression)  and  SLA  =  0  (Single  Input 
Start),  the  Predecessor  Wait  (Pr.Wt)g  is  set  to  zero,  and  the 
action  continues  at  paragraph  3.1.3b(ll),  Iteration  Count 
Actions . 

(7)  For  all  other  Forward  Progression  cases,  the  Pred  input  status 
is  updated  per  paragraph  3. 1.3b (8);  the  actions  beyond  this 
point  depend  on  the  results  of  the  Pred  Input  Status  Check  in 
paragraph  3.1.3b(9). 

(8)  Pred  Input  Status  Update 

(a)  If  (PDNF)g  =  0  (No  prior  Pred  inputs  yet  received),  set 
(PDNF)g  =  1  and  set  (Pr.Wt)g  =  TIME.V.  (TIME.V  is  the 
current  Simulation  Time.)  The  latter  action  saves  the 
starting  time  for  the  Predecessor  Wait  (Pr.Wt)g  calculation 
per  paragraph  3.1.3b(10).  The  box  is  then  placed  in  "Pred 
Wait"  status  (Box.Stat  =  1). 

(b)  If  (SLA)p  =  2  (Or-type  input),  set  (PDNR)g  =  1  (logical  OR 
input  is  satisfied).  (Note:  (PDNR)g  may  already  =  1,  in 
which  case  it  will  remain  1). 


(c)  If  (SLA)p  =  1  (And-type  input),  Set  (PDN)n  =  1. 


(9)  Pred  Input  Status  Dependent  Actions 

If  all  Pred  Input  Status  conditions  are  found  to  be  satisfied 
(i.e.,  (PDNR)g  =  1  and  (PDN)n,g  =  1  for  all  values  of  "n")  for 
an  S-Box,  the  box  is  set  up  for  activation  per  paragraphs  3.1.3b 

(10)  -  (13);  otherwise  the  program  returns. 

(10)  Set  Pred  Wait  (Pr.Wt)  as  follows: 

(Pr.Wt)g  =  TIME . V  -  (Pr.Wt)g.  (See  paragraph  3. 1 . 3b(8) (a) ) . 

(11)  Iteration  Count  (It.Ct)g  Actions 

(a)  Step  (It.Ct)g 

(b)  If  (It.Ct)g  remains  within  limits  (i.e.,  is  less  than 

It. Limit),  continue  with  paragraph  3.1.3.b(12);  otherwise 
the  following  error  message  is  output,  and  exit  occurs. 

IT. CT  HIGH,  (Box. ID),  (DTN.No),  (TIME.V) 

(12)  Internal  Wait  Processing 

Internal  Waits  reflect  inherent  delays  in  starting  a  function 
(e.g.,  the  time  between  when  a  document  is  ready  for  submittal 
until  it  is  in  the  hands  of  the  reviewers).  It  therefore 
applies  each  time  the  box  is  entered.  If  no  Internal  Wait  is 
required  (In.Wt  =  0)  the  Event  Notice  time  (TIME. A)  is  set  to 
"now"  (TIME.V)  and  the  action  skips  to  paragraph  3.1.3b(13). 
Otherwise: 

(a)  The  Wait  Duration  (I. Wait)  is  set  equal  to  the  box 
parameter  In.Wt.  In  future  Simulator  versions,  In.Wt  will 
be  subjected  to  random  perturbation  before  each  use. 

(b)  The  Wait  is  then  added  to  the  Cumulative  Internal  Wait; 
(Cln.Wt)g  =  (Cln.Wt)g  +  (I. Wait). 

(c)  The  box  is  put  into  Internal  Wait  status;  (Box.Stat)g  =  2. 

(d)  If  the  Progression  Mode  is  Forward  (PMN  =  0)  the  Earliest 
Start  Time  is  set  to  now  ((EST)g  =  TIME.V). 

(e)  The  Event  Notice  time  (TIME. A)  is  set  to  its  value  for  the 
end  of  the  Wait  (TIME.V  +  I. Wait). 

(13)  Event  Notice  Generation 

A  Box.Proc  Event  Notice  is  created  and  scheduled  as  follows: 

(a)  The  Event  Time  (TIME. A)  contains  the  value  established  in 
paragraph  3.1.3b(12). 
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(b)  The  Box. Point  designates  the  S-Box  being  processed  (i.e., 
(Box.Pointer)s) . 

(c)  The  Integration  Group  number  (DTNN)  is  set  to  the  value  of 
DTN.No  as  obtained  in  paragraph  3.1.3b(3). 

(d)  The  Progression  Mode  (PMN)  contains  the  value  obtained  in 
paragraph  3.1.3b(4). 

3.1.4  Input  Event  Notice  Parameters 

a.  Box. Point:  This  points  to  the  terminating  box. 

b.  DTNN:  This  was  the  Integration  Group  number  (DTN.No)  for  the 
terminating  box. 

c.  PMN:  This  was  the  Progression  Mode  (PMN)  of  the  terminating  box. 

d.  XIT:  This  was  the  exit  selected  by  the  terminating  box. 

3.1.5  Data  Items  Set  or  Used 

a.  Own. Box  Data 

(1)  (Box.Stat)g  -  Box  status 

(2)  (CMN)a,i  -  Contractor  manpower 

(3)  (GMN)a,i  -  Government  manpower 

(4)  Box. Type  -  Type  of  box 

(5)  (Box. Pointer )s  -  From  selected  Successor  List  (YList  or  NList) . 

b.  Successor  Box  (S-Box)  Data 

( 1 )  Box . Type 

(2)  DTN  -  Group  Dependency 

(3)  PList  -  Predecessor  List  items 

(a)  (SLA)p  Start  Logic  Assigned 

(b)  (Gp.Ctrl)p  DTN.No  continuity  controller 

(c)  (PMA)p  Progression  Mode  assignment 


(4)  DTN.No  -  Group  Number 

(5)  (FPDN)g  -  First  Pred  Input  Received  indicator 


(6)  (RPDN)g  -  Logical  "OR"  Pred  Input  Received  indicator 

(7)  (PDN)n,g  -  Pred  Input  Status  indicator  vector 

(8)  (Pr.Wt)g  -  Predecessor  Wait 

(9)  In.Wt  -  Assigned  Internal  Wait 

(10)  (Cln.Wt)g  -  Cumulative  Internal  Wait 

(11)  (Box.Stat)g  Box  Status 

(12)  (It.Ct)g  Iteration  Count. 

c.  Local  Data 

(1)  Own . Box  -  Pointer  to  terminating  box  address 

(2)  PMP  -  Progression  Mode  of  terminating  box 

(3)  PMN  -  Progression  Mode  of  S-Box 

(4)  PDTN  -  Group  Number  of  terminating  box 

(5)  XIT  -  Successor  list  selection  indicator 

(6)  "p"  -  Own. Box  position  on  PList 

(7)  "n"  -  Own. Box  location  on  the  Predecessor  Input  Status  vector 

(8)  IWait  -  Duration  of  Internal  Wait. 

d.  System  Items 

(1)  TIME.V  -  Current  Simulation  Time 

(2)  (CMNA)a  -  Total  contractor  manpower  use  vector 

(3)  (GMNA)a  -  Total  government  manpower  use  vector 

(4)  Rsr .History  -  Resource  Usage  History  file 

(5)  It. Limit  -  Iteration  count  limit. 

3.2  Box.Proc:  Box  Processor 

3.2.1.  Purpose 

The  Box.Proc  Event  Routine  performs  all  the  actions  needed  to  simulate 
the  functions  represented  by  the  box  addressed  for  this  scheduled  Event. 

Since  these  functions  are  somewhat  dependent  on  the  type  of  box  being  entered, 
both  common  and  box- type -unique  functions  are  processed. 


3.2.2.  Event  Scheduling 

A  Box.Proc  Event  Notice  is  scheduled  by  the  Flow  Event  Routine  whenever 
all  entry  qualifications  for  a  box  have  been  met,  and  after  the  Internal  Wait 
(if  any)  has  been  observed. 

3.2.3  Actions 

a.  Override  Stop 

The  Flow. Override  field  is  checked  to  see  if  a  Flow  Stop  is  indicated 
(Flow. Override  =1).  If  so,  the  program  returns  with  no  further  action. 
Otherwise  the  action  continues  as  follows. 

b.  The  following  initiating  Event  Notice  parameters  are  saved: 

(1)  Box. Point  becomes  the  pointer  (Own. Box)  to  the  box  to  be 
processed. 

(2)  DTNN  becomes  the  Integration  Group  Number  (DTN.No)g  for  this 
access . 

(3)  PMN  is  sa.ed  to  indicate  whether  the  Progression  Mode  is  Forward 
or  Iterative.  This  is  needed  for  any  subsequent  Flow  Event 
Notice  created  per  paragraph  3.2.5. 

c.  Event  Time  Storage 

(1)  If  this  is  a  first  entry  to  the  box  ((It.Ct)g  =  1)  the  Earliest 

Start  Time  (EST)g  is  set  to  now  (TIME.V). 

(2)  In  all  cases,  the  Latest  Finish  Time  (LFT)g  is  set  to  now 

(TIME.V). 


Function 


lZ2£ 
2  or  3 

5 

6 
7 

8  or  9 
others 


(Activity) 

(Decision) 

(Counter) 

(Special  Event  -  Milestone) 
(Special  Event  -  Other) 
(Undefined) 


Paragraph 

e 

f 

g 

h 

TBD 


e.  Activity  Box  Processing 

(1)  Override  Skip  Actions 

If  the  Flow. Override  field  indicates  that  the  box  is  to  be 
skipped  (i.e.,  =  2),  an  immediate  Flow  Event  Notice  (TIME. A  = 
TIME.V)  is  scheduled  per  paragraph  3.2.3e(5),  with  XIT  =  0. 
Otherwise : 

(2)  The  box  is  put  into  "ON"  state  ((Box.Stat)g  =  4). 

(3)  The  duration  of  the  activity  (IDurrn)  is  computed.  On  the 
initial  program  this  is  set  equal  to  the  Assigned  Duration 
(IDur)g, i . 

(4)  IDurrn  is  added  to  the  (cumulative)  Total  Duration  (ADurl)g. 

(5)  The  Latest  Finish  Time  (LFT)g  is  set  to  TIME.V  +  IDurrn. 

(6)  Each  element  of  the  contractor  manpower  use  vector  (CMNU)a  and 
the  government  manpower  use  vector  (GMNU)a  are  computed.  In 
Model  0  these  are  set  equal  to  input  items  (CMN)a,i  and  (GMN)a,i 
respectively. 

(7)  The  assigned  manpower  use  values  ((GMNU)a  and  (GMNU)a)  are  added 
to  the  network  totals  ((CMNA)a  and  (GMNA)a).  These  are  then 
stored  (with  TIME.V)  in  (Rsr .History) . 

(8)  The  man-days  expended  by  the  contractor  (CMND)a  and  by  the 
government  (GMND)a  are  computed;  i.e.: 

(CMND)a  =  (CMNU)a  x  IDurrn 

(GMND)a  =  (GMNU)a  x  IDurrn 

(9)  The  man-day  expenditures  are  added  to  the  cumulative  totals  for 
the  box  (CME)a,g  and  (GME)a,g;  i.e.: 


188 


(CME)a ,g  =  (CME)a ,g  +  (CMND)a 
(GME)a,g  =  (GME)a,g  +  (GMND)a 

(10)  (CMND)a  and  (GMND)a  are  added  to  the  (cumulative)  full  system 
man-day  totals  (FNETCM)a,m  and  (FNETGM)a,m,  respectively.  This 
requires  that  the  man-days  be  assigned  to  one  (or  more)  of  the 
monthly  time  slots  starting  with  the  award  of  the  contract.  If 
more  than  one  month  is  spanned,  the  man-days  are  assigned 
proportionate  with  the  relative  occupancy  of  each  included 
month. 

(11)  If  Own. Box  belongs  to  a  Subnetwork,  the  monthly  distributed 
values  of  man-days  (per  Step  10)  are  added  to  the  Subnetwork 
manpower  use  monthly  totals  (SNETCM)a,m  and  (SNETGM)a ,m. 

(12)  If  this  is  an  iterative  reentry  to  this  box  ((It.Ct)g  GT  1),  the 
iterated  cumulative  totals  are  updated;  i.e.: 

(a)  Add  IDurrn  (duration)  to  (ADur2)g. 

(b)  Add  (CMND)a  (contractor  man-days)  to  (CME2)a,g. 

(c)  Add  (GMND)a  (government  man-days)  to  (GME2)a,g. 

(13)  A  Flow  Event  Notice  is  created  per  paragraph  3.2.5  below. 

(a)  Event  Time  (TIME. A)  is  set  equal  to  now  (TIME.V)  plus  the 
activity  duration  (IDurrn). 

(b)  Box  Exit  (XIT)  is  set  to  zero, 
f.  Decision  Box  Processing 

(1)  Override  Exit  Selection 

(a)  If  Flow. Override  =  2  ("Yes"  exit)  set  XIT  =  0. 

(b)  If  Flow. Override  =  3  ("No"  exit)  set  XIT  =  1. 

(c)  For  the  above  cases,  the  action  continues  at  step  (3). 

(d)  Otherwise,  normal  processing  continues  at  step  (2). 

(2)  Normal  Exit  Resolution 

(a)  The  exit  probability  parameter  (pYES)i  is  used  as  input  to 
the  SIMSCRIPT  II. 5  RANDOM  function  to  select  the  exit 
(XIT). 

(b)  The  exit  selected  (XIT)  is  saved  for  statistical  processing 
as  (XIT)g, i . 
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(3)  A  Flow  Event  Notice  is  created  per  paragraph  3.2.5. 

(a)  Event  time  (TIME. A)  is  set  to  now  (TIME.V). 

(b)  Box  exit  (XIT)  is  set  to  the  value  obtained  in  step  (1)  or 
Step  (2),  as  applicable. 

(4)  The  Box  status  is  set  to  OFF  (Box.Stat)g  =  0. 

g.  Counter  Box  Processing 

(1)  Progression  Mode  check  -  Counter  Box  actions  are  to  be  taken 
only  when  the  box  is  entered  during  Forward  Progression.  If  the 
box  is  addressed  on  Iterative  Progression  (PMN  =  1),  the  Ever 
Routine  returns  without  taking  any  further  action. 

(2)  Override  Exit  selection  -  This  is  the  same  as  for  the  ordinary 
Decision  Box  per  paragraph  3.2.3f(l). 

(3)  Normal  Exit  Resolution 

(a)  The  Counter  Box  Integration  Group  type  attribute  (DTK) 
establishes  the  Group  (1  =  DIG,  2  =  TIG)  to  which  the  box 
belongs.  Based  on  the  DTN  field  content,  the  maximum  Group 
number  (DTN. Max)  is  set  equal  to  CPCI  attribute  NDIG  or 
NTIG.  (These  are  both  input  parameters.) 

(b)  The  Integration  Group  number  DTNN  provided  within  the  Event 
Notice  is  then  compared  with  DTN. Max  to  select  a  flow  exit 
(XIT)  as  follows: 

-  If  DTNN  =  DTN. Max,  set  XIT  =  0("Yes"  exit). 

-  If  DTNN  LT  DTN. Max,  set  XIT  =  l("No"  exit). 

-  Otherwise,  produce  an  Error  Halt  giving: 

Box. ID,  Box. Type,  DTNN  value,  and  Current 
Time  (TIME.V). 

(4)  A  Flow  Event  Notice  is  created  per  paragraph  3.2.5: 

(a)  Event  time  (TIME. A)  is  set  to  now. 

(b)  Box  Exit  (XIT)  is  set  to  the  value  obtained  in  step  (2)  or 
(3),  as  applicable. 

(c)  In  the  event  of  an  Error  Halt  per  paragraph  (3)(b),  no 
Event  Notice  is  created. 

(5)  The  box  is  put  into  OFF  status  (Box.Stat)g  =  0. 

h.  Special  Event  -  Milestone  Processing. 

(1)  A  Flow  Event  Notice  is  created  per  paragraph  3.3.5 


(a)  Event  tine  (TIME-)  is  set  to  now 

(b)  Box  Exit  (XIT)  is  set  to  zero 

(c)  Progression  Mode  (PMN)  is  set  to  forward. 


3.2.4  Input  Event  Notice  Parameters 

a.  Box. Point  -  Designates  the  box  involved 

b.  DTNN  -  Integration  Group  number  for  this  event 

c.  PMN  -  Progression  Mode;  Forward  or  Iterative. 

3.2.5  Output  Event  Notice  Parameters 
(For  a  subsequent  Flow  Event) 

a.  TIME. A  -  Scheduled  event  time 

b.  Box. Point  -  Designates  Own. Box  address 

c.  DTNN  -  Integration  Group  number  output  (always  the  samer  as 
Integration  Group  number  input). 

d.  PMN  -  Progression  Mode  (always  the  same  as  its  value  on  input  Event) 

e.  XIT  -  Exit  selected. 

3.2.6  Data  Item  Set  Or  Used 
a.  Own  Box  Table  Data 

(1)  Common  Items 

(a)  (Box. Type)  -  Box  function 

(b)  (Box.Stat)g  -  Box  status 

(c)  (DTN)  -  Group  type 

(d)  (DTN. No)  -  Integration  Group  number 

(e)  (It.Ct)g  -  Iteration  count 

(f)  (Flow. Override)  -  Override  Indicator 

(g)  (EST)g  Earliest  Start  (or  Occurrence)  Time 

(h)  (LFT)g  -  Latest  Finish  (or  Occurrence)  Time 

(2)  Activity  Box  Items 
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(a)  (IDur)g,i  -  Activity  duration;  input 

(b)  (ADurl)g  -  Activity  duration;  cumulative  total 

(c)  (ADur2)g  -  Activity  duration;  cumulative;  iteration  only 

(d)  (CMN)a,g,i  -  Assigned  manpower;  contractor;  input 

(e)  (GMN)a,g,i  Assigned  manpower;  government;  input 

(f)  (CMEl)a,g  -  Expended  manpower;  contractor;  total  man-days 

(g)  (CME2)a,g  -  Expended  manpower;  contractor;  iterated  man- 

days 

(h)  (GMEl)a,g  -  Expended  manpower;  government,  total  man-days 

(i)  (GME2)a,g  -  Expended  manpower;  government;  iterated  man- 

days  . 

(3)  Decision  Box  Data 

(a)  (pYES)i  -  Yes  exit  probability;  input 

(b)  (XIT)g,i  -  Exit  selection,  saved. 

b.  System  Data 

(1)  (TIME.V)  -  Current  Simulation  Time  (now). 

(2)  (NDIG)  -  Quantity  of  DIGs,  this  CPCI 

(3)  (NTIG)  -  Quantity  of  TIGs,  this  CPCI. 

c.  Local  Data 

(1)  Common  Items 

(a)  (DTNN)  -  DTN  number 

(b)  (PMN)  -  Progression  Mode 

(c)  (XIT)  -  Exit  selected 

(2)  Activity  Box  Items 

(a)  (IDurrn)  -  Activity  duration,  computed  (days) 

(b)  (CMNU)a  -  Manpower  use  rate,  contractor 

(c)  (GMNU)a  -  Manpower  use  rate,  government 
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(d)  (CMND)a  -  Manpower  expended,  man-days,  contractor 

(e)  (GMND)a  -  Manpower  expended,  man- days ,  government. 

(3)  Other  Box  Type  Item 

(a)  DTN.Max  -  The  maximum  quantity  of  this  group. 

Network  Data 

(1)  (FNETCM)a.m  -  Total  men  being  used,  per  month,  full  network, 
contractor 

(2)  (FNETGM)atm  -  Total  men  being  used,  per  month,  full  network, 
government 

(3)  (SNETCM)a ,m  -  Total  men  being  used,  per  month,  each  Subnetwork, 
contractor 

(4)  (SNETGM)a ,m  -  Total  men  being  used,  per  month,  each  Subnetwork, 
government 

(5)  (Rsr.History)a  -  Total  Men  in  use  now;  (sequential  file). 
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APPENDIX  G 


DEMONSTRATION  MODEL  (MO) 
STATISTICS  DEVELOPMENT 


To  produce  the  planned  Simulator  reports,  the  following  data  will  be 
collected  during  each  Single  Simulation  Pass  (SPASS)  and  aggregated  for  the 
many  SPASS  repetitions  to  obtain  statistical  results.  Paragraph  1  of  this 
appendix  defines  the  SPASS  gathered  data;  Paragraph  2  defines  the  SPASS 
repetition  data;  Paragraph  3  briefly  describes  the  content  of  the  output 
report  produced;  and  Paragraph  4  identifies  the  data  to  be  entered  by  the  user 
in  order  to  conduct  a  Simulation  run.  All  Integration  Group  (DTN)  dependent 
data  will  be  collected  per  Group  (DIG  or  TIG),  unless  otherwise  indicated. 

Data  marked  with  an  asterisk  (*)  may  not  be  obtained  until  the  Prototype  Model 
(Ml). 


1.  SPASS  DATA  COLLECTION 
1.1  Activity  Box  (A. Box)  Data 

The  following  collected  for  each  Activity  Box. 

a.  Wait  Durations 

(1)  Pr.Wt  -  Time  spent  waiting  for  the  box’s  predecessors  (Preds). 

It  starts  when  the  first  Pred  is  completed,  and  ends  when  the 
last  required  Pred  is  completed. 

(2)  CIn.Wt  -  The  cumulative  sum  of  all  Internal  Waits  (In.Wts) 
sustained  by  the  box.  Each  Internal  Wait  period  reflects  a 
delay  in  starting  that  is  inherent  in  the  function.  Each  In.Wt 
begins  when  all  Pred  input  requirements  are  satisfied  and  ends 
when  the  specified  Internal  Wait  period  completes.  This  only 
applies  to  boxes  on  which  In.Wt  is  not  zero.  The  full  value  of 
In.Wt  is  applied  to  each  DTN  and  to  each  iteration. 

(3)  *RWtl  -  The  sum  of  all  Waits  that  occur  when  an  activity  is 
otherwise  able  to  begin,  but  cannot  because  manpower  or  other 
needed  resources  are  not  available  (e.g.,  the  development  or 
test  facilities  are  currently  fully  used,  or  are  "down"). 

(4)  *RWt2  -  Same  as  RWtl,  except  that  only  the  Waits  associated  with 
iterative  reentry  are  included. 

b.  Activity  Duration 

(1)  Total  (ADurl)  -  The  time  during  which  an  activity  is  in  actual 
operation.  It  startB  immediately  after  all  Waits  have  completed 
and  continues  until  the  activity  is  ended.  Each  time  the  A. Box 
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is  reentered  for  iteration,  the  incremental  duration  is  added  to 
the  accumulated  duration. 

(2)  Iterative  (ADur2)  -  This  is  the  same  as  ADurl  except  that  only 
the  iterative  reentry  durations  are  accumulated. 

c.  Event  Times 

(1)  Earliest  Start  Time  (EST)  -  The  earliest  time  at  which  the 
function  could  begin,  assuming  no  constraints  other  than 
completion  of  all  required  predecessors. 

(2)  Latest  Finish  Time  (LFT)  -  The  latest  time  at  which  the  activity 
ends.  If  the  Activity  Box  is  entered  several  times  (e.g.,  for 
iteration)  LFT  will  retain  the  simulated  time  when  the  'last 
activation  ends. 

d.  Manpower  Use  Data 

(1)  (CMEl)a  -  The  total  contractor  manpower  used  in  man-days  per 
manpower  type  is  collected  for  each  Activity  Box.  This  includes 
the  sum  of  both  the  first  use  and  all  iterative  reentry  uses. 

(2)  (CME2)a  -  This  is  the  same  as  (CMEl)a  except  that  only  the 
iterative  reentry  use  is  included. 

(3)  (GMEl)a  -  The  same  as  (CMEl)a  except  that  it  applies  to  each 
government  manpower  type. 

(4)  (GME2)a  -  Same  as  (GMEl)a,  except  that  only  iterative  reentry 
use  is  included. 

e.  Iteration  Data 

(1)  It.Ct  -  The  number  of  times  that  the  box  was  activated  (first 
operation  plus  each  subsequent  iterative  operation) . 

1.2  Decision  (Branch)  Box  (B.Box)  Data 

The  following  information  is  collected  for  each  branching  Decision  Box. 

a.  Wait  Duration 

(1)  Pr.Wt  -  Same  as  paragraph  l.la(l). 

(2)  CIn.Wt  -  Same  as  paragraph  l.la(2). 

b.  Event  Times 

(1)  EST  -  Same  as  paragraph  l.lc(l). 
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(2)  LFT  -  The  time  of  last  entry  into  this  Decision  Box;  e.g.,  as  a 
result  of  iteration.  If  the  box  is  entered  only  once,  LFT  = 
EST. 

c.  Iteration  Data 

(1)  It.Ct  -  Same  as  paragraph  l.le(l) 

(2)  (XIT)i  -  The  exit  selected  ("Yes"  or  "No")  on  each  iteration 
(contains  room  for  up  to  "n"  binary  choices,  where  "n"  is  the 
maximum  number  of  allowable  iterations). 

1.3  Counter  Box  (C.Box)  Data 


The  following  information  will  be  collected  for  each  counter  Decision 

Box. 


EST  -  Each  time  the  C.Box  is  entered  the  Simulation  Time  (TIME.V)  is 
retained.  Same  as  EST,  paragraph  1.2b(l). 

1.4  Subprocess  Box  Data 

In  Model  0  a  Subprocess  Box  is  the  same  as  Activity  Box.  The  box 
distinction  is  intended  for  possible  future  use. 

1.5  Support  (Helper)  Box  Data 

This  is  the  same  as  an  Activity  Box.  The  distinction  is  intended  for 
future  use. 

1.6  Special  Event  Box  (E.Box)  Data 

The  following  information  will  be  collected  for  each  Special  Event  Box. 

a.  Milestone  Type 

(1)  EST  -  Records  the  Simulation  Time  (TIME.V)  of  each  entry.  Same 
as  EST,  paragraph  1.2b(l).  (Milestone  Special  Event  Boxes 
should  not  be  entered  via  iteration). 

b.  Other  Types  (TBD) . 

1.7  Subnetwork  Data 


The  following  information  will  be  collected  for  each  of  the  15  possible 
Subnetworks  definable  by  the  user. 

a.  Manning  and  Manpower  Use  -  This  will  aggregate  the  total  manning  and 
manpower  use  by  all  activities  within  the  network,  as  follows: 

(1)  (CSNETMM)a  -  The  contractor  manpower  usage  (in  man-days)  on  each 
Subnetwork  for  each  manpower  type  accumulated  for  each  month 
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(i.e.,  aggregates  of  20  working  days)  frost  start  of  contract. 
Values  for  any  activities  which  span  month  boundaries  will  be 
proportionately  subdivided  among  all  such  months. 

(2)  (GSNETMM)a  -  Same  as  (CSNETMM)a,  except  that  they  cover 
government  manpower  types. 

b.  *0ther  Resource  Utilization  -  The  total  utilization  of  each  defined 
support  resource  (e.g.,  programming  and  test  facilities)  will  be  accumulated 
on  a  monthly  basis . 

1.8  Full  Network  Data 

a.  The  data  defined  for  Subnetworks  (see  paragraph  1.7)  will  be 
collected  for  the  entire  network. 

b.  (CMNA)a  -  A  list  of  all  contractor  personnel  manpower  usage 
transactions  will  be  collected. 

c.  (GMNA)a  -  Same  as  (CMNA)a,  except  for  government  personnel. 


2.  PASS  REPETITION  DATA 

At  the  conclusion  of  each  SPASS  the  box-related  data  retained  per 
paragraphs  1.1  -  1.8  above  will  be  aggregated  to  obtain  statistical  results, 
as  follows: 

2.1  General  Box-related  Data 

Each  of  the  SPASS  data  items  identified  in  the  following  table  will  be: 

a.  summed  to  obtain  its  total;  and 

b.  squared  and  summed  to  obtain  the  basis  for  its  variance: 


Par.  it 

Element 

Data  Name 

Definition 

l.la(l) 

A.  Box 

Pr.Wt 

Predecessor  Wait  Time 

(2) 

II 

CIn.Wt 

Process  Wait  Time 

(3) 

If 

*RWtl 

Resource  Wait  Time  (Total) 

(A) 

II 

*RWt2 

Resource  Wait  Time  (Iterative) 

l.lb(l) 

ADurl 

Total  Activity  Duration 

(2) 

II 

ADur2 

Iterated  Activity  Duration 

l.lc(l) 

•  1 

EST 

Earliest  Start  Time 

(2) 

II 

LFT 

Latest  Finish  Time 

l.ld(l) 

II 

(CMEl)a 

Total  Contractor  Manpower  Usage 

(2) 

II 

(CME2)a 

Iterated  Contractor  Manpower  Usage 

(3) 

M 

(GMEl)a 

Total  Gov't  Manpower  Usage 
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Par.  # 

Element 

Data  Name 

Definition 

(4) 

fl 

(GME2)a 

Iterated  Gov't  Manpower  Usage 

l.le(l) 

•1 

It.Ct 

Iteration  Count 

1.2a(l) 

B.Box 

Pr.Wt 

Predecessor  Wait  Time 

(2) 

II 

CIn.Wt 

Decision  Wait  Time 

1.2b(l) 

»• 

EST 

First  Time  of  Decision 

(2) 

tl 

LFT 

Last  Time  of  Decision 

1 . 2c(l) 

tl 

It.Ct 

Iteration  Count 

1.3 

C.Box 

EST 

Count  Increment  Time 

1 .6a 

E.Box 

EST 

Time  of  Event 

1 . 7a(l) 

SUBNET 

(CSNETMM)a 

Contractor  Manpower 

(2) 

SUBNET 

(GSNETMM)a 

Government  Manpower 

1.8a(l) 

FULNET 

Contractor  Manpower 

(2) 

FULNET 

Government  Manpower 

Idition, 

a  count  of  the  number 

of  times  each  box  was  entered  for  the 

first  or  only  time  (i.e.,  ignoring  iterative  reentry)  will  be  maintained. 

2.2  Decision  Box  Exit  Summaries 

SPASS  data  will  be  processed  to  extract  exit  history  for  each  B.Box  as  a 
function  of  the  iteration  number,  based  on  (XIT)i  data  per  paragraph  1.2c(2); 
e.g.  : 


Iteration  No.  (It.Ct)  12345 
Quantity  of  Exits  100  69  35  6  0 

"Yes"  Exit  Probability  .31  .49  .82  1.00 

2.3  Subnetwork  and  Full  Network  Data 


These  data  will  be  extracted  as  follows: 
a.  Timing  Data 

The  Earliest  Start  Time  (EST)  and  Latest  Finish  Time  (LFT)  (per 
Subnetwork)  obtained  in  each  SPASS  will  be  summed  to  obtain  each  total,  and 
squared  and  summed  to  obtain  each  variance.  Also,  the  most  extreme  values 
(earliest  and  latest)  of  each  will  be  retained. 
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b .  Manpower  Data 


(1)  The  contractor  manpower  expended  (in  man-months  per  month, 
(CSNETMM)a)  will  be  summed,  and  squared  and  summed  to  obtain  the 
total  and  the  variance. 

(2)  The  government  manpower  expended  (GSNETMM)a  will  also  be  treated 
as  in  (1)  above. 

(3)  Corresponding  manpower  level  statistics  will  be  developed  from 
(CFNETMM)a  and  (GFNETMM)a. 

(4)  Statistics  on  total  manpower  expended  (in  man-months)  for  each 
type  of  contractor  and  government  manpower  will  be  obtained  from 
the  corresponding  monthly  totals  for  the  whole  contract  period. 

(5)  *A  profile  of  daily  manpower  use  (by  type)  will  be  obtained  by 
converting  CMNA  and  GMNA  transactions  into  daily  totals. 

c.  *Dollar  Cost  Data 

(.1)  Manpower  dollar  cost  data  will  be  derived  for  each  manpower 

category  listed  in  paragraph  2.3b  above,  by  the  application  of 
entered  labor  rate  values.  Overhead  and  General  & 

Administrative  rates  will  also  be  reflected. 

(2)  *Other  resource  dollar  costs  will  be  treated  in  later  versions. 

(3)  Total  monthly,  annual,  and  full  project  dollar  costs  will  also 
be  developed. 


3.  REPORTS 

Reports  that  include  means  and  standard  deviations,  based  on  the 
accumulations  defined  above,  the  corresponding  box  entry  counts,  and 
appropriate  box,  Subnetwork,  and  full  network  identification  will  be  prepared 
at  the  end  of  each  Simulator  run  (i.e.,  after  all  repetitions).  A  simple 
project  schedule  that  includes  defined  milestones  will  also  be  prepared. 

These  reports'  formats  are  to  be  determined. 


4.  SIMULATOR  INPUT  DATA 

The  Simulator  Model  0,  which  will  be  used  for  project  cost  and  schedule 
forecasting,  will  require  the  following  data  to  be  entered  for  each  CPCI  being 
simulated. 

a.  The  network  linkage  definition  per  Table  B-l; 

b.  Override  information  which  will  customize  the  network  configuration  to 
reflect  the  specific  procurement  condition;  (This  would  be  accomplished  by  the 
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addition  and  deletion  of  function  boxes  and  by  the  corresponding  Modification 
of  the  Predecessor  and  Successor  lists  for  the  remaining  function  boxes.) 


c.  Parameter  data  per  Tables  B-2  through  B-4  that  is  customized  to 
include  any  new  boxes  added  per  step  b  and  by  value  changes  that  reflect  the 
specific  procurement  (including  manpower  category  changes,  if  applicable); 

d.  The  quantity  of  DIGs  planned  for  the  development,  and  the  proportion 
(of  the  effort)  consumed  on  each  DIG; 

e.  The  quanitity  of  TIGs  planned  for  the  qualification  testing  and  the 
proportion  of  effort  consumed  on  each  TIG;  and 

f.  Miscellaneous  project  data  that  includes  the  project  starting  date 
its  estisiated  duration,  and  the  number  of  simulation  repetitions  to  be 
conducted. 
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APPENDIX  H 


DATA  REPORTING  FOR 
LATER  SIMULATOR  MODELS 


1.  STRUCTURES  OF  INTEREST 

Following  are  several  Process  Model  structures  about  which  users  may  want 
information,  and  for  which  the  Process  Model  simulation  program  (i.e.,  the 
Simulator)  must  therefore  collect  statistics.  Activity  Boxes,  Decision  Boxes 
and  Special  Event  boxes  are  the  basic  elements  from  which  the  others  are 
composed.  Subprocess  Calls  are  each  shorthand  notation  for  a  group  of  these 
basic  elements.  Paths,  Subnetworks,  and  the  entire  process  are  respectively 
higher-level  aggregates  of  basic  elements  and  Subprocess  Calls.  Activity 
Boxes,  Decision  Boxes,  Special  Event  Boxes  and  Subnetworks  have  been  explained 
elsewhere  in  this  report.  Subprocesses  and  Paths  are  explained  below. 

A  Subprocess  is  any  portion  of  the  Model  logic  that  (1)  comprises  a 
connected  set  of  two  or  more  basic  elements  or  Subprocess  Calls  (to  the  same 
or  to  another  Subprocess);  and  that  (2)  has  a  single  entrance  and  a  single 
exit.  A  Subprocess  Call  is  any  element  of  the  Model  logic  that  stands  for  an 
instance  of  a  specified  Subprocess.  A  Subprocess  Call  is  exactly  like  a 
closed  subroutine  call  in  a  programming  language.  Like  a  subroutine  call,  a 
Subprocess  Call  may  have  input  or  output  parameters. 

A  Path  is  a  specific  single-thread  sequence  of  Activity  Boxes,  Decision 
Boxes,  and  Subprocess  Calls.  A  Path  can  be  defined  by  a  list  of  connected 
Activity  Boxes,  Subprocess  Call  boxes,  and  specific  Decision  Box  exits.  A 
Path  may  include  no  concurrent  activities  and  may  include  only  a  single  exit 
from  each  of  its  Decision  Boxes.  A  Path  may  include  a  loop.  Examples  of 
Paths  (see  Figure  A-2)  include: 

a.  2A,  6F,  6G,  6H,  6I(Y) 

b.  6G,  6H,  61 (N) . 

Example  b.  includes  a  loop. 

Path  statistics  may  help  Simulator  debugging  and  may  also  help  reveal 
elapsed  times  and  costs  along  single-thread  sequences.  Ability  to  trace  a 
Path  may  be  important,  too. 

Paths  are  special  cases  of  Subnetworks.  No  special  Path  statistics  will 
be  provided,  since  a  Subnetwork  can  be  defined  to  represent  any  Path. 


2.  SPECIFIC  REPORT  TYPES 

The  information  contents  of  the  reports  desired  from  Simulator  Models  1, 
2,  &  3  are  outlined  below,  as  are  the  input  data  needed  to  specify  them. 
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These  reports'  exact  formats  have  yet  to  be  defined.  Asterisks  distinguish 
the  different  Models’  report  contents.  Those  that  Model  1  should  provide  are 
not  flagged.  The  additional  reports  or  report  components  that  are  candidates 
for  Model  2  are  marked  by  a  single  asterisk  (*),  while  those  that  Model  3  or 
later  will  provide  are  flagged  by  two  asterisks  (**) .  The  same  conventions 
apply  to  the  inputs  needed.  Most  of  the  reports  identified  below  are  tabular. 
Graphic  equivalents  of  most  of  these  will  be  provided,  some  as  early  as 
Model  1.  Superimposed  plots  of  selected  simulation  results,  of  the  same  or 
different  type  (if  coi?o?tible)  from  the  same  or  successive  simulation  runs 
will  also  be  available. 


2.1  For  the  Overall  Process 


2.1.1  General 


2. 1.1.1  Inputs  Needed 

a.  Number  of  Simulator  Repetitions 

Note:  This  is  the  number  of  times  that  Simulator  execution  will 
repeat,  with  the  same  inputs  but  with  different  pseudo-random  number 
seeds,  to  smoothe  out  random  errors.  In  Model  2  or  later  we  hope  to 
include  an  adaptive  algorithm  that  will  check  for  convergence  as  a 
function  of  defined  statistical  confidence  limits,  and  that  will  stop 
repetition  sooner  if  and  when  simulation  results  within  the  limits 
have  been  achieved. 


b. 


c . 


Program  Start  Date  (Month,  Day  &  Year) 

System  Structure  Indicators 

(1)  Number  of  System  Segments* 

(2)  Number  of  CPCIs 

(3)  Number  of  Design  Integration  Groups  (DIGs)  per  CPCI 

(4)  Number  of  Test  Integration  Groups  (TIGs)  per  CPCI 

(5)  Number  of  CPCs  per  CPCI 

(6)  Number  of  new  CPCs  per  DIG 

(7)  Intra-CPCI  Dependencies  at  the  DIG  Level. 

Note:  E.g.,  that  DIG-II  of  CPCI  1  may  not  begin  until  DIG-I  of 
CPCI  3  is  finished. 


d.  System  Definition  Matrix** 

Note:  The  system  definition  matrix  will  replace  the  system  structure 
indicators  (input  c.)  in  Simulator  Model  2  and  later.  The  Simulator 
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use  of  the  system  definition  data  (e.g.,  in  parametric  equations) 
will  explicitly  reflect  differences  in  computer  program  size, 
difficulty,  development  approach,  etc. 

(1)  Rows:  System  Components 

Note:  The  system  itself,  each  of  its  segments,  each  of  its 
segment's  CPCIs,  and  each  CPCI's  CPCs  are  the  system  components. 

(2)  Columns:  System  Component  Identifier;  DIG  Number  of  First 
Development;  First  DIG  Development  Priority  (1  =  High);  Test 
Priority  (1  =  High);  SARE  Parameter  Data  Types;  Dependencies. 

Note:  The  system  component  identifier  specifies  each  system 
component's  level  and  hierarchial  position.  The  DIG  number  of 
first  development  specifies  when  that  system  component  will 
first  be  started.  The  first  TIG  number  specifies  when  the 
system  component  will  first  be  tested.  The  SARE  parameters 
specify  each  system  component's  presumed  size,  difficulty,  etc. 

e.  Manpower  Availability  Matrix* 

(1)  Rows:  Months  Since  Program  Start 

(2)  Columns;  Contractor  &  Government  Labor  Categories 

(3)  Elements:  Number  of  Persons  Available 

Note:  Some  future  Simulator  version  may  want  to  represent 
manpower  that  is  somewhat  interchangeable  by  type  (as  people 
really  are).  This  may  entail  representing  numbers  of  people  who 
have  several  skills,  and  assigning  to  a  task  any  person  who  has 
such  a  skill  (subject,  perhaps,  to  assignment  priorities).  This 
outline  does  not  reflect  this  concept  further. 

f.  Other  Resource  Availability  Matrix* 

(1)  Rows:  Months  Since  Program  Start 

(2)  Columns:  Other  Resource  Types 

(3)  Each  Element: 

(a)  Other  Resource  Quantity 

(b)  Quantity's  Unit  of  Measure. 

g.  Integration  Tree  Matrix** 

(1)  Rows:  System  Components  &  Other  Integrands 
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Note :  "Other  Integrands"  =  modules  and  interim  results  of 
integration  that  are  not  system  components. 

(2)  Columns:  TIGs 

(3)  Each  Element: 

(a)  Component  ID  of  Integration  Result 

(b)  Component  IDs  of  All  Integrands. 

h.  Simulation  Report  Selection  Matrix** 

(1)  Rows:  The  Entire  System;  All  Segments;  Each  Segment;  All  CPCIn; 
Each  CPCI ;  All  CPCs;  Each  CPC 

(2)  Columns:  System  Component  Level;  System  Component  Name;  The 
Report  Types  Appropriate  to  Each  Row 

(3)  Each  Element:  Level  Number;  System  Component  Name  (or  "All"); 
Whether  to  Produce  the  Report 

Note:  Many  of  the  reports  identified  below  will  output  data  for 
each  of  the  acquisition  program's  calendar  months  during  which 
manpower  or  other  resources  are  spent.  Simulator  Model  1  will 
assume  a  fixed  number  of  working  days  (i.e.,  20)  per  calendar 
month.  Subsequent  more  sophisticated  versions  (i.e.,  Model  2  or 
later)  may  develop  results  per  a  user-specified,  variable  number 
of  working  days  per  month.  If  so,  the  Model  input  should 
include  the  following  vector: 

i.  Program  Month  Composition  Vector** 

(1)  Rows:  Calendar  Months  Since  Program  Start 

(2)  Each  Element:  Number  of  Working  Days  in  this  Month  (or  Number 
of  Hours  Worked  /  8)  (Precision  =  .1  day). 


2. 1.1.2  Output  Produced 

a.  Listing  of  All  Simulation  Program  Inputs  as  Entered 


b.  See  Sections  2.1.2  -  2.1.7  below. 


2.1.2  Manpower  Use 


2. 1.2.1  Additional  Inputs  Needed 


a.  nl  =  Number  of  Standard  Deviations  (Sigma)  for  Optimistic  Estimates 


b.  n2  =  Number  of  Standard  Deviations  (Sigma)  for  Pessimistic  Estimates 
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2 . 1 . 2 . 2  Output  Produced 


a.  Mean  (Expected  Value)  Manpower  Expenditure  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Manpower  Types;  Contractor,  Government,  &  Grand  Totals 

Note:  The  Simulator  will  be  designed  so  that  the  numbers  of 
contractor  and  government  manpower  types ,  as  well  as  their 
identities,  can  easily  be  changed.  Need  for  such  alteration  is 
likely  as  the  Simulator's  application  changes. 

(3)  Each  Element:  Mean  Number  of  Man-Days  Expended  (in  All 
Repetitions) . 

b.  Manpower  Expenditure  Variance  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Manpower  Types;  Contractor,  Government,  &  Grand  Totals 

(3)  Each  Element:  Variance  of  Number  of  Man-Days  Spent 

Note:  We  currently  foresee  four  sources  of  manpower  variation: 

(1)  different  numbers  of  initial  activity  executions  due  to 
random  decision  box  alternative  selection;  (2)  different  numbers 
of  iterations  due  to  random  loop  control  decision  outcomes;  (3) 
in  Model  3  and  later,  different  manpower  expenditure  per 
activity  and  per  activity  iteration  due  to  random  sampling  of 
manpower  level  and  duration  based  3-value  (PERT-like)  estimates; 
and  (4)  differences  in  monthly  values  due  to  the  different 
delays  that  result  from  the  abvove  factors  and  different 
manpower  constraints. 

c.  Optimistic  Manpower  Use  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Manpower  Types;  Contractor,  Government,  &  Grand  Totals 

(3)  Each  Element:  (Mean  -  nl  Sigma)  Number  of  Man-Days  Spent. 

d.  Pessimistic  Manpower  Use  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Manpower  Types;  Contractor,  Government,  &  Grand  Totals 

(3)  Each  Element:  (Mean  +  n2  Sigma)  Number  of  Man-Days  Spent. 


Note:  Corresponding  reports  showing  manning  levels  (vice  man- 
days)  will  also  be  provided. 

2.1.3  Other  Resources  Use* 

2. 1.3.1  Additional  Inputs  Needed* 

a.  n3  =  Number  o£  Standard  Deviations  (Sigma)  for  Optimistic  Estimates* 

b.  n4  =  Number  of  Standard  Deviations  (Sigma)  for  Pessimistic 
Estimates*. 

2. 1.3. 2  Output  Produced* 

a.  Mean  (Expected  Value)  Other  Resource  Use  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Resource  Types  and  Units  of  Measure 

(3)  Each  Element:  Expected  Quantity  of  this  Resource  Used. 

b.  Other  Resource  Use  Variance  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Resource  Types  and  Units  of  Measure 

(3)  Each  Element: 

(a)  Number  of  Initial  Activity  Executions  Entailing  Use  of  this 
Resource  Type  During  this  Month. 

(b)  Variance  of  Quantity  of  this  Resource  Used. 

c.  Optimistic  Other  Resource  Use  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Resource  Types  and  Units  of  Measure 

(3)  Each  Element:  (Mean  -  n3  Sigma)  Quantity  of  Each  Resource  Used. 

d.  Pessimistic  Other  Resource  Use  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Resource  Types  and  Units  of  Measure 

(3)  Each  Element:  (Mean  +  n4  Sigma)  Quantity  of  Each  Resource  Used. 
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2.1.4  Dollar  Cost 


2. 1.4.1  Additional  Inputs  Needed 

a.  Assumed  Salary  +  Overhead  Rate  Matrix 

(1)  Rows:  Months  Since  Program  Start 

Columns:  Contractor  and  Government  Labor  Categories 


(2) 

(3) 


Elements:  Undiscounted  Hourly  Labor  Rate  (Including  Overhead) 
for  Each  Month  Since  Program  Start 

Note:  The  proposed  reports  do  not  distinguish  between  regular 
and  overtime  pay.  To  do  so  the  Simulator  would  need  to  handle 
separate  rates  and  manhour  estimates  for  both. 

b.  Assumed  Other  Resource  Rate  Matrix* 

(1)  Rows:  Months  Since  Program  Start 

(2)  Columns:  Resource  Types 

(3)  Each  Element:  Undiscounted  Rate  per  Unit  of  this  Resource  Type 

Note:  These  units  must  be  the  same  as  those  specified  for  in 
the  other  Resource  Availability  Matrix  (see  paragraph  2. 1.1. If). 

Note :  The  Simulator  (Model  2  and  later)  should  calculate  costs 
as  of  the  valuation  date  (input  c.).  Using  the  Assumed  Interest 
Rate  Vector  (input  d.),  the  costs  of  work  performed  before  the 
Valuation  Date  should  be  accumulated  (i.e.,  increased)  and  the 
costs  of  work  to  be  performed  after  the  valuation  date  should  be 
discounted  (i.e.,  decreased). 

c.  Valuation  Date  (Month  &  Year). 

d.  Assumed  Interest  Rate  Vector 

Note :  The  Simulator  provides  for  input  of  an  interest  rate  vector  to 
reflect  the  effect  of  anticipated  changes  in  interest  rate.  However, 
the  Simulator  should  optionally  accept  a  single  rate  and  create  a 
(constant)  vector  from  it. 

Note:  To  develop  costs  that  ignore  interest,  the  user  should  input 
no  Assumed  Interest  Rate  Vector. 

(1)  Rows:  Months  Since  Program  Start 

(2)  Each  Element:  Assumed  Annual  Simple  Interest  Rate  Applicable  to 
the  Month. 
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2 . 1 . 4 . 2  Output  Produced 


a.  Mean  (Expected  Value)  Manpower  Cost  Matrix 

(1)  Rows:  Calendar  Months  Since  Prograai  Start;  FY  &  Grand  Totals 

(2)  Columns:  Manpower  Types;  Contractor,  Govenuaent,  &  Grand  Totals 

(3)  Each  Element:  Accumulated  or  Discounted  Dollar  Value  as  of 
Valuation  Date  of  Mean  Manpower  of  this  Type  Expended. 

b.  Optimistic  Manpower  Cost  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Manpower  Types;  Contractor,  Government,  &  Grand  Totals 

(3)  Each  Element:  Accumulated  or  Discounted  Dollar  Value  as  of 
Valuation  Date  of  (Mean  -  nl  Sigma)  Manpower  of  this  Type 
Expended . 

c.  Pessimistic  Manpower  Cost  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Manpower  Types;  Contractor,  Government,  &  Grand  Totals 

(3)  Each  Element:  Accumulated  or  Discounted  Dollar  Value  as  of 
Valuation  Date  of  (Mean  +  n2  Sigma)  Manpower  of  this  Type 
Expended . 

d.  Mean  (Expected  Value)  Other  Resource  Cost  Matrix* 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Resource  Types;  Government  &  Contractor  Totals;  Grand 
Total 

(3)  Each  Element:  Accumulated  or  Discounted  Dollar  Value  as  of 
Valuation  Date  of  Mean  Other  Resource  Use. 

e.  Optimistic  Other  Resource  Cost  Matrix* 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Resource  Types;  Government  &  Contractor  Totals;  Grand 
Total 

(3)  Each  Element:  Accumulated  or  Discounted  Dollar  Value  as  of 
Valuation  Date  of  (Mean  -  n3  Sigma)  Other  Resource  Use. 

f.  Pessimistic  Other  Resource  Cost  Matrix* 


(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Resource  Types;  Government  &  Contractor  Totals;  Grand 
Total 

(3)  Each  Element:  Accumulated  or  Discounted  Dollar  Value  as  o£ 
Valuation  Date  of  (Mean  +  n4  Sigma)  Other  Resource  Use. 

g.  Mean  (Expected  Value)  Total  Dollar  Cost  Matrix* 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Contractor  &  Government  Totals;  Grand  Total 

(3)  Each  Element:  Sum  of  the  Corresponding  Column  from  the  Mean 
Manpower  Cost  and  Other  Resource  Cost  Matrices. 

h.  Optimistic  Total  Dollar  Cost  Matrix* 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Contractor  &  Government  Totals;  Grand  Total 

(3)  Each  Element:  Sum  of  the  Corresponding  Columns  from  the 
Optimistic  Manpower  Cost  and  Other  Resource  Cost  Matrices. 

i.  Pessimistic  Total  Dollar  Cost  Matrix* 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns:  Contractor  &  Government  Totals;  Grand  Total 

(3)  Elements:  Sum  of  the  Corresponding  Columns  from  the  Pessimistic 
Manpower  Cost  and  the  Optimistic  Other  Resource  Cost  Matrices. 

2.1.5  Elapsed  Time 

2. 1.5.1  Additional  Inputs  Needed 

a.  Gantt  Chart  Content  &  Arrangement  Option(s) 

(l)  Chart  Content  Options 

(a)  Milestones  (&  Corresponding  Box  IDs)  Only 

(b)  Milestones  &  Subnetworks  Only 

(i)  All  Milestones  &  Subnetworks 

(ii)  List-specified  Subset  Only 

(c)  Milestones  Plus  Activity  &  Decision  Boxes  Only* 
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(i)  All  Activities  &  Decisions 

(ii)  List-specified  Subset  Only 

(d)  Milestones,  Subnetworks,  &  Component  Activities  & 
Decisions* 

(i)  All  Milestones,  Subnetworks,  &  Component  Activities  & 
Decisions 

(ii)  List-specified  Subset  Only 

(2)  Line  Content  Options 

(a)  Mean  Initial  Start  Time  to  Finish  Time 

(b)  Four  Components* 

(i)  Wait  for  Immediate  Predecessors 

(ii)  Internal  Wait  Time 

(iii)  Wait  for  Resources 

(iv)  Activity  Duration 

(3)  Arrangement  Options 

(a)  Network  Definition  Table  Order 

(b)  By  Subnetwork  Number 

(c)  By  Increasing  Mean  Start  Time  (at  Each  Level  Included)* 

(d)  User-Specified  Order*. 

b.  n5  =  Number  of  Standard  Deviations  for  Optimistic  Estimates 

c.  n6  =  Number  of  Standard  Deviations  for  Pessimistic  Estimates 
2. 1.5.2  Output  Produced 

a.  Gantt  Chart  of  Milestones 

Note:  For  Model  1  this  may  only  be  a  list  of  Milestones  and 

associated  dates. 

b.  Gantt  Chart  of  Milestones  Plus  Mean  Start  &  Finish  Times. 


c.  Gantt  Chart  of  Milestones  Plus  Optimistic  Start  &  Finish  Times. 

d.  Gantt  Chart  of  Milestones  Plus  Pessimistic  Start  &  Finish  Tiows. 
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e.  Gantt  Chart  of  Milestones  Plus  Optimistic,  Mean,  &  Pessimistic  Start 
&  Finish  Times.* 

f.  Network  Logic  Diagram  Showing  Milestones  Plus  Boxes  Annotated  by 
Optimistic,  Mean,  &  Pessimistic  Start  &  Finish  Times.**  The  diagram  will 
highlight  the  critical  path  through  the  network. 

2.1.6  Resource  Constraint  History* 

2. 1.6.1  Additional  Inputs  Needed* 

a.  n7  =  First  Sampling  Day 

b.  n8  =  Sampling  Interval 

Note :  The  Resource  Constraint  History  Tables  (See  paragraph 
2.1.6.2a)  present  values  of  resources  provided,  in  use,  in  demand  but 
unavailable,  etc.,  as  of  a  specified  sample  of  days  since  acquisition 
program  start.  n7  and  n8  (both  integers)  specify  the  first  such  day 
and  the  number  of  days  between  successive  samples. 

2. 1.6.2  Output  Produced* 

a.  Resource  Constraint  History  Tables  (One  per  Manpower  or  Other 
Resource  Type) 

(1)  Once-per-Table  Data 

(a)  Resource  Type  Identifier 

(b,c)Maximum  &  Minimum  Value  of  Resource  Provided  (Rl) 

(d,e)Maximum  &  Minimum  Value  of  Resource  In  Use  (R2) 

(f , g)Maximum  &  Minimum  Value  of  Resource  Unused  (R3) 

(h,i)Maximum  &  Minimum  Value  of  Unsatisfied  Demand  (R4) 

(j,k)Maximum  and  Minimum  Number  of  Processes  in  Execution  (Ne) 

(l,m)Maximum  &  Minimum  Number  of  Processes  Delayed  because  of 
this  Resource  was  Unavailable  (Nd) 

(2)  Time  History  of  Resource  Status 

(a)  Rows:  Selected  Days  Since  Program  Start  (i.e.,  n7,  n7  + 
n8,  n7  +  2n8,  etc.) 

(b)  Columns: 

(i)  Column  1:  Day  Number  of  Sample 
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(ii)  Columns  2-8:  Rl,  R2,  R3,  R4,  R2  +  R3  +  R4;  Ne;  Nd 

(iii) Each  Element:  Value  of  the  Variable  at  Start  of  the 
Day 

Note:  In  Model  2  or  later,  these  tables  should  be  supplemented 
by  plotted  charts. 

b.  Resource  Shortage  Impact  Matrices  (One  per  Manpower  or  Other  Resource 

Type) 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns: 

Note:  Each  column  presents  a  type  of  value  related  to  the 
availability  of  this  resource.  Each  element  shows  the  value  of 
this  type  for  the  time  period  represented  by  the  row.  The 
elements  of  columns  4,  6,  7,  and  8  quantify  time-related  losses 
during  each  such  time  period  that  happen  because  (at  least  in 
part)  a  shortage  of  this  resource  prevents  one  or  more  processes 
from  starting  or  beginning  iteration  during  the  time  period. 

Note:  A  process  may  need  more  than  one  type  of  resource,  and 
may  be  delayed  because  too  little  of  two  or  more  such  resource 
types  are  available.  In  such  cases  the  Simulator  will  collect 
unavailability  data  independently  for  each  such  resource,  during 
whatever  periods  the  resource  is  in  short  supply. 

(a)  Column  1:  Number  of  Resource  Unit-Days  (e.g. ,  Man-Days) 
Provided 

(b)  Column  2:  Number  of  Resource  Unit-Days  Used 

(c)  Column  3:  Number  of  Available  Resource  Unit-Days  Unused 

(d)  Column  4:  Number  of  Unavailable  Resource  Unit-Days 

Note:  This  is  the  sum,  for  each  process  delayed  because  it 
couldn't  get  enough  of  the  resource,  of  each  required 
amount  of  the  resource  times  the  number  of  days  the  process 
was  delayed  because  it  couldn't  get  the  needed  amount. 

(e)  Column  5:  Sum  of  Column  2,  3,  &  4  Values 

Note:  This  may  often  somewhat  exceed  the  Column  1  value. 

(f)  Column  6:  Number  of  Different  Processes  Delayed 

(g)  Column  7:  Number  of  Process-Days  Lost 


212 


Note:  This  is  the  sum  of  the  delays  incurred  during  the 
period  by  all  processes  delayed  (during  the  period)  because 
(in  part  at  least)  they  couldn't  get  enough  of  the 
resource. 

(h)  Column  8:  Normalized  Size  of  the  Shortage 

Note :  Column  4/Column  7.  This  is  a  rough  estimate  of  the 
level  of  this  resource  needed  over  the  entire  period  to 
allay  the  shortage.  However,  users  should  note  that 
providing  this  level  may  not  entirely  relieve  the  problem, 
in  part  because  of  interactions  among  this  and  other 
constraints.  E.g.,  relieving  another  bottleneck  could 
increase  the  demand  on  this  resource. 

c.  Resource  Demand  Lists* 

(1)  For  any  Manpower  or  Other  Resource  Type;  and  for  any  month,  any 
fiscal  year,  or  the  entire  acquisition  program  duration,  a  list 
of  the  processes  consuming  the  resource,  and  the  amount  consumed 
during  the  period,  may  be  requested. 

(2)  A  similar  list  of  the  processes  unable  to  start  or  repeat  during 
the  period  because  they  cannot  obtain  enough  of  the  resource  may 
be  requested.  Both  lists  will  be  arranged  in  decreasing  order 
of  Resource  Unit-Days.  These  lists  should  help  identify  prime 
"cost-drivers" . 

d.  Predecessor  Completion  Impact  Matrix 

(1)  Rows:  Calendar  Months  Since  Program  Start;  FY  &  Grand  Totals 

(2)  Columns: 

(a)  Column  1:  Number  of  Different  Processes  Delayed  Waiting 
for  Essential  Immediate  Predecessors  to  Finish 

(b)  Column  2:  Number  of  Process-Days  Lost  Thereby 

(c)  Column  3:  Mean  Number  of  Process-Days  Lost  (i.e.,  Column 
2/Column  1) 

Note :  This  report  is  intended  to  indicate: 

(1)  limits  on  the  benefits  of  overlapping  concurrent  activities  (for 
a  given  process); 

(2)  the  potential  for  improvement  by  process  revisions  aimed  at 
better  matching  the  durations  of  concurrent  process  paths. 
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2.1.7  Trace 


SIMSCRIPT  II. 5  provides  a  listing  of  each  Event  as  it  is  removed  from  the 
Event  queue.  It  also  provides  a  program  flow  trace  (from  the  main  program) 
when  a  program  aborts.  Both  of  these  look  useful. 

2.2  For  Each  Subnetwork 

2.2.1  Additional  Inputs  Needed 

a.  Subnetwork  Membership  of  Each  Basic  Element  &  Subprccess  Call. 

b.  Same  as  Overall  System  Inputs  (see  paragraph  2.1.1). 

2.2.2  Output  Produced 

Same  as  Overall  System  Results  (see  paragraph  2.1.2). 

2.3  For  Each  Path 

No  output  will  be  provided  for  Paths  per  se,  since  a  user  can  designate 
each  Path  of  interest  as  a  Subnetwork. 

2.4  For  Each  Activity  Box 
2.4.1  Additional  Inputs  Needed 

a.  Activity  Box  ID. 

b.  Subnetwork  Number  (0  =  None). 

c.  Subprocess  ID  (Blank  =  None). 

d.  Activity  Priority  (1  =  High).* 

e.  Responsible  Agency  Code  (i.e.,  "Doer"). 

f.  System  Level  Code. 

g.  Number  of  Immediate  Predecessors. 

h.  Immediate  Predecessor  Matrix  (Same  as  paragraph  2.4.2a(l)(i) . 

i.  Immediate  Successor  Vector 

Each  Element:  Box  ID  of  an  Immediate  Successor 

j .  Internal  Wait  Time 

(1)  Mean  Value 

(2)  Variance. 


k.  Required  Manpower  Matrix  (For  Each  Integration  Group) 

(1)  Columns  =  Manpower  Types;  e.g.,  Contractor  Sys,  Prg,  Tst,  Spt; 
Government  participating  organizations. 

Note:  These  types  should  be  checked  vs.  those  designed  for 

SARE. 

(2)  Rows:  One  per  Iteration 

(3)  Each  Element:  Number  of  Persons  of  This  Type  Needed  per 
Iteration. 

l.  Required  Other  Resource  Matrix  (One  per  Integration  Group)* 

(1)  Columns:  Other  Resource  Types  &  Units 

(2)  Rows:  One  per  Iteration 

(3)  Each  Element:  Number  of  Units  of  this  Resource  Needed  for  this 
Iteration. 

m.  Required  Duration  Vector  (For  each  Integration  Group) 

(1)  One  Element  per  Iteration 

(2)  Each  Element  a  Pair 

(a)  Expected  Value  of  Duration 

(b)  Variance  of  Duration. 

n.  Duration  Iteration  Vector 

(1)  One  Element  per  Iteration 

(2)  Each  Element  the  Proportion  of  the  Original  Duration  Estimate 
Required  to  Complete  Each  Iteration. 

2.4.2  Output  Produced 

a.  Activity  Box  Activation  Tables  (One  Set  per  Activity  Box.  One 
Instance  per  Integration  Group,  &  Total). 

(1)  Once  per  Table  Data: 


(a)  Activity  Box  ID 

(b)  Subnetwork  Number  (0  =  None) 

(c)  Containing  Subprocess  ID  (Blank  =  None) 


(d)  Activity  Priority  (1  =  High)* 

(e)  Integration  Group  Number;  "Total" 

(f)  Responsible  Agency  Code  (i.e.,  "Doer") 

(g)  System  Level  Code 

(h)  Number  of  Immediate  Predecessors 

(i)  Immediate  Predecessor  Matrix 

(i)  Rows:  One  per  Immediate  Predecessor 

(ii)  Columns:  Box  ID  (including  Decision  Box  Exit  ID);  Box 
Type;  Start  Logic  (see  Figure  A-l);  Progression  Mode; 
Group  Number  Controller;  Iteration  Loop  Return? 

(Yes,  No);  Integration  Group  Repetition  Return?  (Yes, 
No). 

(j)  Number  of  Immediate  Predecessors  Requiring  Initial  Wait 

(k)  Mean  Initial  Wait  for  Required  Immediate  Predecessors 

(l)  Variance  of  Initial  Wait 

(m)  Mean  Initial  Internal  Wait  Time 

(n)  Initial  Internal  Wait  Time  Variance 

(o)  Number  of  Different  Manpower  &  Other  Resource  Types  Needed 

(p)  Mean  Wait  for  Available  Manpower  &  Other  Resources 

(q)  Variance  of  Wait  for  Available  Resources 

(r)  Mean  Initial  Activity  Start  Time 

(s)  Variance  of  Initial  Activity  Start  Time 


(t)  Mean  Total  Activity  Duration:  All  Iterations 

(u)  Variance  of  Total  Activity  Duration:  All  Iterations 

(v)  Mean  Total  Elapsed  Time  ((m)+(p)+(t)) 

(w)  Variance  of  Total  Elapsed  Time 

(x)  Mean  Activity  Finish  Time  ((r)+(v)) 

(y)  Variance  of  Activity  Finish  Time 


(z)  Activity  Manpower  Use  Vector 

One  Element  per  Manpower  Type.  Each  Element:  Total 
Manpower  of  the  Type  Used. 

(aa)  Activity  Manpower  Use  Variance  Vector 

One  Element  per  Manpower  Type.  Each  Element:  Variance  of 
Total  Manpower  of  this  Type  Used 

(ab)  Other  Resource  Use  Vector* 

One  Element  per  Other  Resource  Type.  Each  Element:  Total 
of  the  Other  Resource  Type  Used;  Units 

(ac)  Other  Resource  Use  Variance  Vector* 

t One,  Element  per  Other  Resource  Type.  Each  Element: 

Variance  of  the  Total  Other  Resource  Type  Used 

(ad)  Mean  Number  of  Iterations 

(ae)  Variance  of  Number  of  Iterations 
(2)  Per  Iteration  Data 

Note:  Each  time  a  box  is  activiated,  per  Simulator  repetition 
and  per  Integration  Group,  is  an  iteration. 

(a)  Rows: 

(i)  Number  of  Activations,  this  Iteration  (for  All 
Simulator  Repetitions) 

(ii)  Relative  Frequency  of  Activation,  this  Iteration 
(i.e.,  Column  (i)  Value  /  Actual  Number  of  Simulator 
Repetitions) 

(iii) Mean  Activity  Duration,  this  Iteration 

(iv)  Activity  Duration  Variance,  this  Iteration 

(v)  Mean  Activity  Manpower  Use  Vector,  this  Iteration  (One 
Element  per  Manpower  Type) 

(vi)  Activity  Manpower  Variance  Vector 

(vii)  Mean  Activity  Other  Resource  Use  Vector,  this 
Iteration*  (One  Element  per  Other  Resource  Type) 

(viii)Activity  Other  Resource  Use  Variance  Vector 
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(b)  Columns:  One  per  Iteration. 

2.5  For  Each  Decision  Box 

2.5.1  Additional  Inputs  Weeded 

a.  Decision  Box  ID. 

b.  Subnetwork  Number  (0  =  None). 

c.  Subprocess  ID*  (Blank  =  None). 

d.  Responsible  Agency  Code  (i.e.,  "Doer"). 

e.  System  Level  Code. 

f.  Number  of  Immediate  Predecessors. 

g.  Immediate  Predecessor  Matrix  Same  as  paragraph  2.4.2a(l)(i) . 

h.  Possible  Decision  Box  Outcome  Matrix 

(1)  Rows:  Possible  Branches 

(2)  Columns: 

(a)  Column  1:  Branch  ID 

(b)  Column  2:  Expected  Branch  Probability,  First  Iteration 

(c)  Columns  3-n  (n  less  than  11):  Expected  Branch  Probability, 
Subsequent  Iterations. 

i.  Immediate  Successor  Matrix 

(a)  Rows:  Possible  Branches 

(b)  Columns:  Possible  Immediate  Successors  of  Each  Branch 

(c)  Each  Element:  Branch  ID;  Immediate  Successor  ID. 

2.5.2  Output  Produced 

a.  Decision  Box  Activation  Table  (One  Set  per  Decision  Box.  One 
Instance  per  Integration  group,  &  Total). 

(1)  Once  per  Table  Data: 

(a)  Decision  Box  ID 

(b)  Subnetwork  Number  (0  =  None) 
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(c)  Containing  Subprocess  ID*  (Blank  =  None) 


(d)  Integration  Group  Number;  "Total" 

(e)  Responsible  Agency  Code  (i.e.,  "Doer") 

(f)  System  Level  Code 

(g)  Number  of  Immediate  Predecessors 

(h)  Immediate  Predecessor  Matrix  (Same  as  paragraph  2.4.2a(l)) 

(j)  Number  of  Initial  Predecessors  Requiring  Initial  Wait 

(k)  Mean  Initial  Wait  for  Required  Immediate  Predecessors 

(l)  Variance  of  Initial  Wait 

(m)  Mean  Time  of  First  Occurrence 

(n)  Variance:  Time  of  First  Occurrence 

(o)  Mean  Number  of  Iterations 

(p)  Variance  of  Number  of  Iterations. 

(2)  Per  Iteration  Data 

Note :  Each  time  a  box  is  activated,  per  simulation  program 
repetition  and  integration  group  is  an  iteration. 

(a)  Rows: 

(i)  Number  of  Activations,  this  Iteration  (for  All 
Simulation  Program  Repetitions) 

(ii)  Relative  Frequency  of  Activation,  this  Iteration 
(i.e.,  Column  (i)  Value  /  Actual  Number  of  Simulator 
Repetitions) 

(iii) ID  of  First  Branch 

(iv)  Expected  Probability  of  Selection  (p(E)):  First 
Branch 

(v)  Relative  Frequency  of  Selection  (p(A)):  First  Branch 

(vi)  Variance  of  Relative  Selection  Frequency:  First 
Branch 

(vii) ID,  p(E),  p(A),  &  Variance*:  Second  Branch 


I 

i 
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(viii)Corresponding  Values  for  any  Third,  Fourth,  etc., 
Branches 

(b)  Columns:  One  per  Iteration 

Note:  These  per  iteration  data  should  help  the  user  decide 
how  closely  the  Monte  Carlo  simulation  of  branch  selection 
follows  the  probabilities  expected. 

2.6  For  Each  Special  Event  Box 

2.6.1  Additional  Inputs  Needed 


a. 

Special  Event  Box  ID. 

b. 

Subnetwork  Number  (0  =  None-) . 

c. 

Subprocess  ID  (Blank  =  None). 

d. 

Type  of  Special  Event  (1  ~  Milestone; 

2  =  Random  Process  Modifier) 

e. 

System  Level  Code. 

f. 

Number  of  Immediate  Predecessors. 

g- 

Immediate  Predecessor  Matrix  (Same  as 

paragraph  2.4.2a(l) (i)) . 

h. 

Immediate  Successor  Vector 

Each  Element:  Box  ID  of  an  Immediate 

Successor. 

i. 

Boxes  Altered  Vector 

j- 

Parameter  Vector 

2.6.2  Output  Produced 

a.  Special  Box  Activation  Table  (One  Set  per  Activity  Box.  One  Instance 
per  Integration  Group  &  Total). 

(1)  Once  per  Table  Data: 

(a)  Special  Event  Box  ID 

(b)  Subnetwork  Number  (0  =  None) 

(c)  Containing  Subprocess  ID*  (Blank  =  None) 

(d)  Special  Event  Type  (1  =  Milestone;  2  =  Process  Modifier) 

(e)  Integration  Group  Number;  "Total" 
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(f)  System  Level  Code 

(g)  Number  of  Immediate  Predecessors 

(h)  Immediate  Predecessor  Matrix  (Same  as  paragraph 
2.4.2a(l)(i)) 

(i)  Number  of  Immediate  Predecessors  Requiring  Initial  Waits 

(j)  Mean  Initial  Wait  for  Required  Immediate  Predecessors 

(k)  Variance  of  Initial  Wait 

(l)  Mean  Initial  Special  Event  Start  Time 

(m)  Variance  of  Initial  Activity  Start  Time* 

(n)  Mean  Number  of  Iterations 

(o)  Variance  of  Number  of  Iterations 
(2)  Per  Iteration  Data 

Note:  Each  time  a  box  is  activated,  per  simulation  program 
repetition  and  per  Integration  Group,  is  an  iteration. 

(a)  Rows: 

(i)  Number  of  Activations,  this  Iteration  (for  All 
Simulation  Program  Repetitions) 

(ii)  Relative  Frequency  of  Activation,  this  Iteration 
(i.e.,  Column  (i)  Value  /  Actual  Number  of  Simulation 
Program  Repetitions) 

(b)  Columns:  One  per  Iteration. 

2.7  For  Each  Subprocess  Call* 

2.7.1  Inputs  Needed* 

Same  as  Outputs  Produced  items  2. 7 . 2a . (1) (a)-(d) ,  etc. 

2.7.2  Output  Produced* 

a.  Subprocess  Call  Activation  Table  (One  Set  per  Subprocess  Call.  One 
Instance  per  Integration  Group  &  Total). 


(1)  Once  per  Table  Data 

(a)  Subprocess  Call  ID 
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(b)  Subnetwork  Number  (0  =  None) 

(c)  Containing  Subprocess  ID  (Blank  =  None) 

(d)  ID  of  Called  Subprocess 

(e)  Priority  of  this  Subprocess  Call  (1  =  High) 

(f)  Subprocess  Call  Input  Parameter  Vector 
Each  Element:  Input  Parameter  ID 

(g)  Subprocess  Call  Output  Parameter  Vector 
Each  Element:  Output  Parameter  ID 

(h)  Same  as  Activity  Box  Results  Required  Items  (l)(e)-(ae) 

Note:  These  results  are  to  be  developed  as  if  the  called 
Subprocess  were  a  single  activity. 

(2)  Per  Iteration  Data* 

Same  as  Activity  Box  Per  Iteration  Data  (2.4.2a. (2)) . 

2.8  For  Each  Subprocess* 

Note:  Each  component  of  a  subprocess  is  presumed  defined  as  a  basic  element 
or  subprocess  call.  See  paragraphs  2.4.1,  2.5.1,  2.6.1  and  2.7.1. 

2.8.1  Inputs  Needed* 

a.  Component  Matrix* 

(1)  Rows:  One  per  Component 

(2)  Columns:  Component  ID;  Component  Type  (i.e.,  Activity  Box, 
Decision  Box,  Special  Event  Box,  Subprocess  Call). 

2.8.2  Output  Produced* 


Same  as  Inputs  Needed. 


GLOSSARY 


AFLC 

AFSC 

ATM 

Box . Proc 

CCI&C 

CDR 

Cl 

CPCI 

CPDP 

CPC 

CPT&E 

CRISP 

CSDCA 

DELIV 

DEV 

DID 

DIG 

DOC 

DSGN 

ECP 

ESD 

FACIL 

FCA 


Acronyms  and  Abbreviations 

Air  Force  Logistics  Command 

Air  Force  Systems  Command 

Activity  Timing  and  Manpower  Data 

The  label  assigned  to  the  Simulator  Event  Notice 
type  which  processes  each  Model  function  box 

Code,  Compile,  Integrate  &  Checkout 

Critical  Design  Review 

Configuration  Item 

Computer  Program  Configuration  Item 

Computer  Program  Development  Plan 

Computer  Program  Component 

Computer  Program  Test  &  Evaluation 

Computer  Resources  Integrated  Support  Plan 

Center  for  Software  Data  Collection  and  Analysis 

Deliver 

Develop 

Data  Item  Description 
Developmental  Integration  Group 
Document 
Designer 

Engineering  Change  Proposal 
Electronic  Systems  Division 
Facility 

Functional  Configuration  Audit 
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r 

; 

i 

1 

GLOSSARY  (Continued) 

f  Flow 

: 

The  label  assigned  to  the  Simulator  Event  Notice 
type  which  controls  box-to-box  transition 

FQT 

Formal  Qualification  Testing 

FSD 

Full-Scale  Development 

HiSim 

High  Simulation  Level  '  j 

I&C 

Integration  and  Checkout 

• 

LoSim 

Low  Simulation  Level 

MGMT 

Management 

No . Succ 

Decision  Box  Successor  List  (if  "No" 

Exit  is  Taken) 

ORG 

Organize 

OTD 

Occurrence  and  Timing  Data 

PCA 

Physical  Configuration  Audit 

PDR 

Preliminary  Design  Review 

PERT 

Program  Evaluation  and  Review  Technique 

PM 

Program  Manager 

PMR 

Program  Management  Review 

PO 

r 

Program  Office 

PQT 

Preliminary  Qualification  Testing  # 

Pred 

Function  Box  Predecessor  List 

PROC 

Procedure  * 

PRGM 

Program 

|  PRGMRS 

Programmers 

PROD 

Produce  1 

PROJ 

Project  i 

i 

L  i ... .. 
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GLOSSARY  (Concluded) 


QUAL 

Qualification 

RFP 

Request  for  Proposal 

SARE 

Software  Acquisition  Resource  Expenditure 

SCEWG 

Software  Cost  Estimation  Working  Group 

SEMP 

System  Engineering  Management  Plan 

SPEC 

Specification 

SPRT 

Support 

SYS 

System 

TAC 

Tactical  Air  Command 

TEMP 

Test  and  Evaluation  Master  Plan 

TIG 

Test  Integration  Group 

TOI 

Computer  Systems  Engineering  Directorate 

Yes.Succ 

Function  Box  Successor  List  (if  "Yes"  or 
Only  Exit  is  Taken) 
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